From d8ca0a2e1ade0bac4a6c7eabe53a47b326dd10bf Mon Sep 17 00:00:00 2001 From: linghengqian Date: Mon, 14 Nov 2022 16:21:38 +0800 Subject: [PATCH] Fixes documentation for building GraalVM Native Image --- .../startup/graalvm-native-image.cn.md | 87 ++++++++++++++---- .../startup/graalvm-native-image.en.md | 92 +++++++++++++++---- 2 files changed, 144 insertions(+), 35 deletions(-) diff --git a/docs/document/content/user-manual/shardingsphere-proxy/startup/graalvm-native-image.cn.md b/docs/document/content/user-manual/shardingsphere-proxy/startup/graalvm-native-image.cn.md index f5446d1e9ac9e..8e52315838341 100644 --- a/docs/document/content/user-manual/shardingsphere-proxy/startup/graalvm-native-image.cn.md +++ b/docs/document/content/user-manual/shardingsphere-proxy/startup/graalvm-native-image.cn.md @@ -11,7 +11,20 @@ weight = 2 ## 注意事项 - ShardingSphere Proxy 尚未准备好与 GraalVM Native Image 集成。 - 其在 https://github.com/apache/shardingsphere/actions/ 存在每日构建的任务用于测试构建。 + 其在 https://github.com/apache/shardingsphere/pkgs/container/shardingsphere-proxy-native 存在每夜构建。 + 假设存在包含`server.yaml` 的 `conf` 文件夹为 `./custom/conf`，你可通过如下的 `docker-compose.yml` 文件进行测试。 + +```yaml +version: "3.8" + +services: + apache-shardingsphere-proxy-native: + image: ghcr.io/apache/shardingsphere-proxy-native:latest + volumes: + - ./custom/conf:/conf + ports: + - "3307:3307" +``` - 若你发现构建过程存在缺失的 GraalVM Reachability Metadata, 应当在 https://github.com/oracle/graalvm-reachability-metadata 打开新的 issue ， @@ -21,8 +34,8 @@ weight = 2 需要等待 Junit 5 Platform 的集成，你总是需要在构建 GraalVM Native Image 的过程中， 加上特定于 `GraalVM Native Build Tools` 的 `-DskipNativeTests` 或 `-DskipTests` 参数跳过 Native Image 中的单元测试。 -- 本节假定处于 Linux（amd64，aarch64）， MacOS（ amd64 ）或 Windows（amd64）环境。 - 如果你位于 MacOS(aarch64/M1) 环境, 你需要关注尚未关闭的 https://github.com/oracle/graal/issues/2666 。 +- 本节假定处于 Linux（amd64，aarch64）， MacOS（amd64）或 Windows（amd64）环境。 + 如果你位于 MacOS（aarch64/M1） 环境，你需要关注尚未关闭的 https://github.com/oracle/graal/issues/2666。 ## 前提条件 @@ -49,21 +62,21 @@ weight = 2 - 在 Git Source 同级目录下执行如下命令, 直接完成 Native Image 的构建。 ```bash -./mvnw -am -pl shardingsphere-distribution/shardingsphere-proxy-native-distribution -B -Pnative -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat.skip=true clean package +./mvnw -am -pl distribution/proxy-native -B -Pnative -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat.skip=true clean package ``` - 情形二：需要使用存在 SPI 实现的 JAR 或 GPL V2 等 LICENSE 的第三方依赖的 JAR。 -- 在 `shardingsphere-distribution/shardingsphere-proxy-native-distribution/pom.xml` 的 `dependencies` 加入存在 SPI 实现的 JAR +- 在 `distribution/proxy-native/pom.xml` 的 `dependencies` 加入存在 SPI 实现的 JAR 或第三方依赖的 JAR。示例如下 ```xml - mysql - mysql-connector-java - 8.0.30 + com.mysql + mysql-connector-j + 8.0.31 org.apache.shardingsphere @@ -76,7 +89,7 @@ weight = 2 - 通过命令行构建 GraalVM Native Image。 ```bash -./mvnw -am -pl shardingsphere-distribution/shardingsphere-proxy-native-distribution -B -Pnative -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat.skip=true clean package +./mvnw -am -pl distribution/proxy-native -B -Pnative -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat.skip=true clean package ``` 3. 通过命令行启动 Native Image, 需要带上两个参数， @@ -84,13 +97,13 @@ weight = 2 假设已存在文件夹`./custom/conf`，示例为 ```bash -./apache-shardingsphere-proxy 3307 ./custom/conf +./apache-shardingsphere-proxy-native 3307 ./custom/conf ``` 4. 如果需要构建 Docker Image, 在添加后存在 SPI 实现的依赖或第三方依赖后, 在命令行执行如下命令。 ```shell -./mvnw -am -pl shardingsphere-distribution/shardingsphere-proxy-native-distribution -B -Pnative,docker.native -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat.skip=true clean package +./mvnw -am -pl distribution/proxy-native -B -Pnative,docker.native -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat.skip=true clean package ``` - 假设存在包含`server.yaml` 的 `conf` 文件夹为 `./custom/conf`，可通过如下的 `docker-compose.yml` 文件启动 GraalVM Native @@ -108,11 +121,49 @@ services: - "3307:3307" ``` -- 如果您使用默认构建配置， 你当然可以为 `shardingsphere-distribution/shardingsphere-proxy-native-distribution/Dockerfile` - 使用 `scratch` 作为 base docker image。 - 但如果您主动为`pom.xml`的`native profile`添加`jvmArgs`为`-H:+StaticExecutableWithDynamicLibC`， - 以静态链接除 `glic` 之外的所有内容，您应该切换 base image 到 `busybox:glic`。 - 参考 https://www.graalvm.org/22.2/reference-manual/native-image/guides/build-static-executables/ 。 - 另请注意，某些第三方依赖将需要更多系统库，例如 `libdl`。 - 因此请确保根据您的使用情况调整 base docker image 和`shardingsphere-distribution/shardingsphere-proxy-native-distribution` +- 如果你不对 Git Source 做任何更改， 上文提及的命令将使用 `oraclelinux:9-slim` 作为 Base Docker Image。 + 但如果你希望使用 `busybox:glic`，`gcr.io/distroless/base` 或 `scratch` 等更小体积的 Docker Image 作为 Base Docker + Image，你需要根据 https://www.graalvm.org/22.2/reference-manual/native-image/guides/build-static-executables/ 的要求， + 做为 `pom.xml`的 `native profile` 添加 `-H:+StaticExecutableWithDynamicLibC` 的 `jvmArgs` 等操作。 + 另请注意，某些第三方依赖将需要在 `Dockerfile` 安装更多系统库，例如 `libdl`。 + 因此请确保根据你的使用情况调整 `distribution/proxy-native` 下的 `pom.xml` 和 `Dockerfile` 的内容。 + +# 可观察性 + +- 针对 GraalVM Native Image 形态的 ShardingSphere + Proxy，其提供的可观察性的能力与 https://shardingsphere.apache.org/document/current/cn/user-manual/shardingsphere-proxy/observability/ + 并不一致。 + +- 你可以使用 https://www.graalvm.org/22.2/tools/ 提供的一系列命令行工具或可视化工具观察 GraalVM Native Image + 的内部行为，并根据其要求使用 VSCode 完成调试工作。 + 如果你正在使用 IntelliJ IDEA 并且希望调试生成的 GraalVM Native Image， + 你可以关注 https://blog.jetbrains.com/idea/2022/06/intellij-idea-2022-2-eap-5/#Experimental_GraalVM_Native_Debugger_for_Java + 及其后继。 + +- 对于使用 `ShardingSphere Agent` 等 APM Java Agent 的情形， + GraalVM 的 `native-image` 组件尚未完全支持在构建 Native Image 时使用 + javaagent，你需要关注尚未关闭的 https://github.com/oracle/graal/issues/1065。 + +- 以下部分采用 `Apache SkyWalking Java Agent` 作为示例，可用于跟踪 GraalVM 社区的对应 issue。 + +1. 下载 https://dlcdn.apache.org/skywalking/java-agent/8.12.0/apache-skywalking-java-agent-8.12.0.tgz ， + 并解压到 ShardingSphere Git Source 的 `distribution/proxy-native`。 + +2. 修改 `distribution/proxy-native/pom.xml` 的 `native profile`， + 为 `org.graalvm.buildtools:native-maven-plugin` 的 `configuration` 添加如下的 `jvmArgs`。 + +```xml + + + -Dskywalking.agent.service_name="your service name" + -Dskywalking.collector.backend_service="your skywalking oap ip and port" + -javaagent:./skywalking-agent/skywalking-agent.jar + +``` + +3. 通过命令行构建 GraalVM Native Image。 + +```bash +./mvnw -am -pl distribution/proxy-native -B -Pnative -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat.skip=true clean package +``` \ No newline at end of file diff --git a/docs/document/content/user-manual/shardingsphere-proxy/startup/graalvm-native-image.en.md b/docs/document/content/user-manual/shardingsphere-proxy/startup/graalvm-native-image.en.md index b253007729b3d..bf482eeb6f439 100644 --- a/docs/document/content/user-manual/shardingsphere-proxy/startup/graalvm-native-image.en.md +++ b/docs/document/content/user-manual/shardingsphere-proxy/startup/graalvm-native-image.en.md @@ -11,7 +11,22 @@ corresponding `Docker Image` through the `native-image` component of `GraalVM`. ## Notice - ShardingSphere Proxy is not yet ready to integrate with GraalVM Native Image. - It has daily build tasks at https://github.com/apache/shardingsphere/actions/ for testing builds. + Fixes documentation for building GraalVM Native Image It exists nightly builds + at https://github.com/apache/shardingsphere/pkgs/container/shardingsphere-proxy-native. + Assuming there is a `conf` folder containing `server.yaml` as `./custom/conf`, you can test it with the + following `docker-compose.yml` file. + +````yaml +version: "3.8" + +services: + apache-shardingsphere-proxy-native: + image: ghcr.io/apache/shardingsphere-proxy-native:latest + volumes: + - ./custom/conf:/conf + ports: + - "3307:3307" +```` - If you find that the build process has missing GraalVM Reachability Metadata, A new issue should be opened at https://github.com/oracle/graalvm-reachability-metadata, @@ -53,22 +68,22 @@ corresponding `Docker Image` through the `native-image` component of `GraalVM`. Image. ```bash -./mvnw -am -pl shardingsphere-distribution/shardingsphere-proxy-native-distribution -B -Pnative -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat.skip=true clean package +./mvnw -am -pl distribution/proxy-native -B -Pnative -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat.skip=true clean package ``` - Scenario 2: It is necessary to use a JAR that has an SPI implementation or a third-party dependent JAR of a LICENSE such as GPL V2. - Add SPI implementation JARs or third-party dependent JARs to `dependencies` - in `shardingsphere-distribution/shardingsphere-proxy-native-distribution/pom.xml`. Examples are as follows + in `distribution/proxy-native/pom.xml`. Examples are as follows ```xml - mysql - mysql-connector-java - 8.0.30 + com.mysql + mysql-connector-j + 8.0.31 org.apache.shardingsphere @@ -81,7 +96,7 @@ corresponding `Docker Image` through the `native-image` component of `GraalVM`. - Build GraalVM Native Image via command line. ```bash -./mvnw -am -pl shardingsphere-distribution/shardingsphere-proxy-native-distribution -B -Pnative -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat.skip=true clean package +./mvnw -am -pl distribution/proxy-native -B -Pnative -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat.skip=true clean package ``` 3. Start Native Image through the command line, you need to bring two parameters, @@ -90,14 +105,14 @@ corresponding `Docker Image` through the `native-image` component of `GraalVM`. Assuming the folder `./custom/conf` already exists, the example is ```bash -./apache-shardingsphere-proxy 3307 ./custom/conf +./apache-shardingsphere-proxy-native 3307 ./custom/conf ```` 4. If you need to build a Docker Image, after adding the dependencies of the SPI implementation or third-party dependencies, execute the following commands on the command line. ```shell -./mvnw -am -pl shardingsphere-distribution/shardingsphere-proxy-native-distribution -B -Pnative,docker.native -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat .skip=true clean package +./mvnw -am -pl distribution/proxy-native -B -Pnative,docker.native -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat .skip=true clean package ``` - Assuming that there is a `conf` folder containing `server.yaml` as `./custom/conf`, you can start the Docker Image @@ -115,11 +130,54 @@ services: - "3307:3307" ``` -- If you use the default build configuration, you can of course use `scratch` as the base docker image - for `shardingsphere-distribution/shardingsphere-proxy-native-distribution/Dockerfile`. - But if you actively add `jvmArgs` to `-H:+StaticExecutableWithDynamicLibC` for the `native profile` of `pom.xml`, - To statically link everything except `glic`, you should switch the base image to `busybox:glic`. Refer - to https://www.graalvm.org/22.2/reference-manual/native-image/guides/build-static-executables/. - Also note that some third-party dependencies will require more system libraries, such as `libdl`. - So make sure to adjust the base docker image and the content of `pom.xml` and `Dockerfile` - under `shardingsphere-distribution/shardingsphere-proxy-native-distribution` according to your usage. +- If you don't make any changes to the Git Source, the commands mentioned above will use `oraclelinux:9-slim` as the + Base Docker Image. + But if you want to use a smaller Docker Image like `busybox:glic`, `gcr.io/distroless/base` or `scratch` as the Base + Docker Image, you need according + to https://www.graalvm.org/22.2/reference-manual/native-image/guides/build-static-executables/, + Add operations such as `-H:+StaticExecutableWithDynamicLibC` to `jvmArgs` as the `native profile` of `pom.xml`. + Also note that some 3rd party dependencies will require more system libraries such as `libdl` to be installed in + the `Dockerfile`. + So make sure to tune `distribution/proxy-native` according to your usage + `pom.xml` and `Dockerfile` below. + +# Observability + +- ShardingSphere for GraalVM Native Image form Proxy, which provides observability capabilities + with https://shardingsphere.apache.org/document/current/cn/user-manual/shardingsphere-proxy/observability/ + Not consistent. + +- You can observe GraalVM Native Image using a series of command line tools or visualization tools available + at https://www.graalvm.org/22.2/tools/, and use VSCode to debug it according to its requirements. + If you are using IntelliJ IDEA and want to debug the generated GraalVM Native Image, + You can + follow https://blog.jetbrains.com/idea/2022/06/intellij-idea-2022-2-eap-5/#Experimental_GraalVM_Native_Debugger_for_Java + and its successors. + +- In the case of using APM Java Agent such as `ShardingSphere Agent`, + GraalVM's `native-image` component is not yet fully supported when building Native Images + javaagent, you need to follow https://github.com/oracle/graal/issues/1065 which has not been closed. + +- The following sections use the `Apache SkyWalking Java Agent` as an example, which can be used to track corresponding + issues from the GraalVM community. + +1. Download https://dlcdn.apache.org/skywalking/java-agent/8.12.0/apache-skywalking-java-agent-8.12.0.tgz and `untar` it + to `distribution/proxy-native` in ShardingSphere Git Source. + +2. Modify the `native profile` of `distribution/proxy-native/pom.xml`, + Add the following `jvmArgs` to the `configuration` of `org.graalvm.buildtools:native-maven-plugin`. + +```xml + + + -Dskywalking.agent.service_name="your service name" + -Dskywalking.collector.backend_service="your skywalking oap ip and port" + -javaagent:./skywalking-agent/skywalking-agent.jar + +``` + +3. Build the GraalVM Native Image from the command line. + +```bash +./mvnw -am -pl distribution/proxy-native -B -Pnative -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotless.apply.skip=true -Drat.skip =true clean package +``` \ No newline at end of file