Fix scripts and optimize docs on version 1.1.2

.github/workflows/build.yml

@@ -24,12 +24,10 @@ on:
 
 jobs:
   build:
-
     runs-on: ubuntu-latest
-
     strategy:
       matrix:
-        node-version: [14.17.3]
+        node-version: [16.13.1]
         # See supported Node.js release schedule at https://nodejs.org/en/about/releases/
 
     steps:

.github/workflows/dead-link-checker.yml

@@ -0,0 +1,17 @@
+name: Dead Link Check
+
+on: [push]
+
+jobs:
+  dead-links-check:
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    if: (github.repository == 'WeBankFinTech/Exchangis')
+    steps:
+      - uses: actions/checkout@v3
+      - uses: gaurav-nelson/github-action-markdown-link-check@v1
+        with:
+          use-quiet-mode: 'no'
+          use-verbose-mode: 'yes'
+          folder-path: '../'
+          config-file: '.github/workflows/dlc.json'

Dockerfile

@@ -1,8 +1,8 @@
 FROM harbor.local.hching.com/library/jdk:8u301
 
-ADD assembly-package/target/wedatasphere-exchangis-1.0.0-RC1.tar.gz /opt/wedatasphere-exchangis.tar.gz
+ADD assembly-package/target/wedatasphere-exchangis-1.1.2.tar.gz /opt/wedatasphere-exchangis.tar.gz
 
-RUN cd /opt/wedatasphere-exchangis.tar.gz/packages/ && tar -zxf exchangis-server_1.0.0-RC1.tar.gz && cd /opt/wedatasphere-exchangis.tar.gz/sbin
+RUN cd /opt/wedatasphere-exchangis.tar.gz/packages/ && tar -zxf exchangis-server_1.1.2.tar.gz && cd /opt/wedatasphere-exchangis.tar.gz/sbin
 
 WORKDIR /opt/wedatasphere-exchangis.tar.gz/sbin
 

README-ZH.md

@@ -6,7 +6,7 @@
 
 ## 介绍
 
-Exchangis 1.0.0 是微众银行大数据平台 WeDataSphere 与社区用户共同研发的的新版数据交换工具，支持异构数据源之间的结构化和非结构化数据传输同步。
+Exchangis是微众银行大数据平台 WeDataSphere 与社区用户共同研发的的新版数据交换工具，支持异构数据源之间的结构化和非结构化数据传输同步。
 
 Exchangis 抽象了一套统一的数据源和同步作业定义插件，允许用户快速接入新的数据源，并只需在数据库中简单配置即可在页面中使用。
 
@@ -16,7 +16,7 @@ Exchangis 抽象了一套统一的数据源和同步作业定义插件，允许
 
 ### 界面预览
 
-![image](https://user-images.githubusercontent.com/27387830/171488936-2cea3ee9-4ef7-4309-93e1-e3b697bd3be1.png)
+![image](images/zh_CN/ch1/frontend_view.png)
 
 ## 核心特点
 
@@ -42,26 +42,29 @@ Exchangis 抽象了一套统一的数据源和同步作业定义插件，允许
 ### 3.  与DSS工作流打通，一站式大数据开发的门户
 
 - 实现DSS AppConn包括一级 SSO 规范，二级组织结构规范，三级开发流程规范在内的三级规范；
-
 - 作为DSS工作流的数据交换节点，是整个工作流链路中的门户流程，为后续的工作流节点运行提供稳固的数据基础；
 
+### 4.  支持多种导数引擎
+
+- 支持Sqoop和DataX引擎进行多种异构数据源之间的导数
+
 ## 整体设计
 
 ### 架构设计
 
-![架构设计](https://user-images.githubusercontent.com/27387830/173026793-f1475803-9f85-4478-b566-1ad1d002cd8a.png)
+![架构设计](images/zh_CN/ch1/home_page_zh.png)
 
 
 ## 相关文档
-[安装部署文档](https://github.com/WeBankFinTech/Exchangis/blob/dev-1.0.0/docs/zh_CN/ch1/exchangis_deploy_cn.md)  
-[用户手册](https://github.com/WeBankFinTech/Exchangis/blob/dev-1.0.0/docs/zh_CN/ch1/exchangis_user_manual_cn.md)
+[安装部署文档](docs/zh_CN/ch1/exchangis_deploy_cn.md)  
+[用户手册](docs/zh_CN/ch1/exchangis_user_manual_cn.md)
 
 ## 交流贡献
 
 如果您想得到最快的响应，请给我们提 issue，或者扫码进群：
 
-![communication](images/zh_CN/ch1/communication.png)
+![communication](images/zh_CN/ch1/code.png)
 
 ## License
 
-Exchangis is under the Apache 2.0 License. See the [License](../../../LICENSE) file for details.
+Exchangis is under the Apache 2.0 License. See the [License](./LICENSE) file for details.

README.md

@@ -4,7 +4,7 @@ English | [中文](README-ZH.md)
 
 ## Introduction
 
-Exchangis 1.0.0 is a new version of data exchange tool jointly developed by WeDataSphere, a big data platform of WeBank, and community users, which supports the synchronization of structured and unstructured data transmission between heterogeneous data sources.
+Exchangis is a new version of data exchange tool jointly developed by WeDataSphere, a big data platform of WeBank, and community users, which supports the synchronization of structured and unstructured data transmission between heterogeneous data sources.
 
 Exchangis abstracts a unified set of data source and synchronization job definition plugins, allowing users to quickly access new data sources and use them on pages with simple configuration in the database.
 
@@ -14,7 +14,7 @@ With the help of [Linkis](https://github.com/apache/incubator-linkis) computing
 
 ###  Interface preview 
 
-![image](https://user-images.githubusercontent.com/27387830/171488936-2cea3ee9-4ef7-4309-93e1-e3b697bd3be1.png)
+![image](images/zh_CN/ch1/frontend_view.png)
 
 ## Core characteristics
 
@@ -47,21 +47,21 @@ With the help of [Linkis](https://github.com/apache/incubator-linkis) computing
 
 ### Architecture Design
 
-![架构设计](images/en_US/ch1/architecture.png)
+![架构设计](images/zh_CN/ch1/home_page_en.png)
 
 
 ## Documents
 
-[Quick Deploy](https://github.com/WeBankFinTech/Exchangis/blob/dev-1.0.0/docs/en_US/ch1/exchangis_deploy_en.md)  
-[User Manual](https://github.com/WeBankFinTech/Exchangis/blob/dev-1.0.0/docs/en_US/ch1/exchangis_user_manual_en.md)
+[Quick Deploy](docs/en_US/ch1/exchangis_deploy_en.md)  
+[User Manual](docs/en_US/ch1/exchangis_user_manual_en.md)
 
 ## Communication and contribution 
 
 If you want to get the fastest response, please mention issue to us, or scan the code into the group ：
 
-![communication](images/en_US/ch1/communication.png)
+![communication](images/en_US/ch1/code.png)
 
 ## License
 
-Exchangis is under the Apache 2.0 License. See the [License](../../../LICENSE) file for details.
+Exchangis is under the Apache 2.0 License. See the [License](./LICENSE) file for details.
 

assembly-package/config/dss-exchangis-server.properties

@@ -0,0 +1,67 @@
+#
+# Copyright 2019 WeBank
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+#
+
+wds.linkis.test.mode=false
+wds.linkis.server.mybatis.datasource.url=jdbc:mysql://{IP}:{PORT}/{database}?useSSL=false&characterEncoding=UTF-8&allowMultiQueries=true&useAffectedRows=true
+wds.linkis.server.mybatis.datasource.username={username}
+wds.linkis.server.mybatis.datasource.password={password}
+
+wds.linkis.gateway.ip={LINKIS_IP}
+wds.linkis.gateway.port={LINKIS_PORT}
+wds.linkis.gateway.url=http://{LINKIS_IP}:{LINKIS_PORT}/
+wds.linkis.log.clear=true
+wds.linkis.server.version=v1
+
+# datasource client
+wds.exchangis.datasource.client.serverurl=http://{LINKIS_IP}:{LINKIS_PORT}/
+wds.exchangis.datasource.client.authtoken.key=EXCHANGIS-AUTH
+wds.exchangis.datasource.client.authtoken.value=EXCHANGIS-AUTH
+wds.exchangis.datasource.client.dws.version=v1
+
+# launcher client
+wds.exchangis.client.linkis.server-url=http://{LINKIS_IP}:{LINKIS_PORT}/
+wds.exchangis.client.linkis.token.value=EXCHANGIS-AUTH
+wds.exchangis.datasource.extension.dir=exchangis-extds/
+
+##restful
+wds.linkis.server.restful.scan.packages=com.webank.wedatasphere.exchangis.datasource.server.restful.api,\
+  com.webank.wedatasphere.exchangis.project.server.restful,\
+  com.webank.wedatasphere.exchangis.job.server.restful
+
+wds.linkis.server.mybatis.mapperLocations=classpath*:com/webank/wedatasphere/exchangis/job/server/mapper/impl/*.xml,\
+classpath*:com/webank/wedatasphere/exchangis/project/server/mapper/impl/*.xml,\
+classpath*:com/webank/wedatasphere/exchangis/project/provider/mapper/impl/*.xml,\
+classpath*:com/webank/wedatasphere/exchangis/engine/server/mapper/*.xml
+
+wds.linkis.server.mybatis.BasePackage=com.webank.wedatasphere.exchangis.dao,\
+  com.webank.wedatasphere.exchangis.project.server.mapper,\
+  com.webank.wedatasphere.exchangis.project.provider.mapper,\
+  com.webank.wedatasphere.linkis.configuration.dao,\
+  com.webank.wedatasphere.linkis.metadata.dao,\
+  com.webank.wedatasphere.exchangis.job.server.mapper,\
+  com.webank.wedatasphere.exchangis.job.server.dao,\
+  com.webank.wedatasphere.exchangis.engine.dao
+
+wds.exchangis.job.task.scheduler.load-balancer.flexible.segments.min-occupy=0.25
+wds.exchangis.job.task.scheduler.load-balancer.flexible.segments.max-occupy=0.5
+#wds.exchangis.job.scheduler.group.max.running-jobs=4
+
+wds.linkis-session.ticket.key=bdp-user-ticket-id
+wds.exchangis.limit.interface.value=false
+
+wds.exchangis.publicKeyStr=
+wds.exchangis.privateKeyStr=

assembly-package/sbin/daemon.sh

@@ -22,6 +22,7 @@ else
   source ./launcher.sh
   source ./common.sh
 fi
+load_env_definitions ${ENV_FILE}
 
 usage(){
   echo "Usage is [start|stop|restart {server}]"

assembly-package/sbin/install.sh

@@ -125,7 +125,7 @@ interact_echo(){
 
 # Initalize database
 init_database(){
-    BOOTSTRAP_PROP_FILE="${CONF_PATH}/exchangis-server.properties"
+    BOOTSTRAP_PROP_FILE="${CONF_PATH}/dss-exchangis-server.properties"
     if [ "x${SQL_SOURCE_PATH}" != "x" ] && [ -f "${SQL_SOURCE_PATH}" ]; then
         `mysql --version >/dev/null 2>&1`
         DATASOURCE_URL="jdbc:mysql:\/\/${MYSQL_HOST}:${MYSQL_PORT}\/${DATABASE}\?useSSL=false\&characterEncoding=UTF-8\&allowMultiQueries=true"
@@ -142,7 +142,7 @@ init_database(){
 }
 
 init_properties(){
-    BOOTSTRAP_PROP_FILE="${CONF_PATH}/exchangis-server.properties"
+    BOOTSTRAP_PROP_FILE="${CONF_PATH}/dss-exchangis-server.properties"
     APPLICATION_YML="${CONF_PATH}/application-exchangis.yml"
     LINKIS_GATEWAY_URL="http:\/\/${LINKIS_GATEWAY_HOST}:${LINKIS_GATEWAY_PORT}\/"
     if [ "x${LINKIS_SERVER_URL}" == "x" ]; then

docs/en_US/ch1/component_upgrade_en.md

@@ -1,5 +1,5 @@
 # Exchangis Component Upgrade Documentation
-This article mainly introduces the upgrade steps for adapting DSS1.1.2 and Linkis1.4.0 on the basis of the original installation of the Exchangis service. The biggest difference between the Exchangis1.0.0 and the Exchangis1.0.0-rc1 version is the installation of the ExchangisAppconn, which needs to be replaced by the entire Exchangisappconn. and load### 1.升级Exchangis前的工作
+This article mainly introduces the upgrade steps for adapting DSS1.1.2 and Linkis1.4.0 on the basis of the original installation of the Exchangis service. The biggest difference between the Exchangis1.1.2 and the Exchangis1.0.0 version is the installation of the ExchangisAppconn, which needs to be replaced by the entire Exchangisappconn. and load### 1.升级Exchangis前的工作
 Before you upgrade Exchangis, please follow the[DSS1.1.2Install and deploy documentation](https://github.com/WeBankFinTech/DataSphereStudio-Doc/tree/main/zh_CN/%E5%AE%89%E8%A3%85%E9%83%A8%E7%BD%B2)
 and [Linkis1.4.0Install and deploy documentation](https://linkis.staged.apache.org/zh-CN/docs/1.4.0/deployment/deploy-quick)Complete the installation and upgrade of DSS and Linkis
 
@@ -13,7 +13,7 @@ Go to the following directory and find exchangis appconn folder and delete:
 ```
 
 #### 2）Download binary package
-We provide the upgrade material package of ExchangisAppconn, which you can download and use directly.[Click to jump Release interface](https://osp-1257653870.cos.ap-guangzhou.myqcloud.com/WeDatasphere/Exchangis/exchangis1.0.0-rc/exchangis-appconn.zip)
+We provide the upgrade material package of ExchangisAppconn, which you can download and use directly.[Click to jump Release interface](https://osp-1257653870.cos.ap-guangzhou.myqcloud.com/WeDatasphere/Exchangis/exchangis1.1.2/Exchangis1.1.2_install_package.zip)
 
 #### 3） Compile and package
 
@@ -72,4 +72,4 @@ After the installation and deployment of exchangis-appconn is completed, you can
 2.	Check whether the project is created synchronously on the exchange side. If the creation is successful, the appconn installation is successful.
 ![image](https://user-images.githubusercontent.com/27387830/169782337-678f2df0-080a-495a-b59f-a98c5a427cf8.png)
 
-For more usage, please refer to[Exchangis1.0 User Manual](https://github.com/WeBankFinTech/Exchangis/blob/dev-1.0.1-rc/docs/zh_CN/ch1/exchangis_user_manual_cn.md)
+For more usage, please refer to[Exchangis User Manual](docs/zh_CN/ch1/exchangis_user_manual_cn.md)

docs/en_US/ch1/exchangis_appconn_deploy_en.md

@@ -3,7 +3,7 @@
 This paper mainly introduces the deployment, configuration and installation of ExchangisAppConn in DSS(DataSphere Studio)1.0.1.
 
 ### 1. Preparations for the deployment of ExchangisAppConn 
-Before you deploy ExchangisAppConn, please follow the [Exchangis1.0.0 to install the deployment document](https://github.com/WeBankFinTech/Exchangis/blob/dev-1.0.0/docs/en_US/ch1/exchangis_deploy_en.md) to complete the installation of Exchangis1.0.0 and other related components, and ensure that the basic functions of the project are available. 
+Before you deploy ExchangisAppConn, please follow the [Exchangis to install the deployment document](docs/en_US/ch1/exchangis_deploy_en.md) to complete the installation of Exchangis and other related components, and ensure that the basic functions of the project are available. 
 
 ### 2. Download and compilation of the ExchangisAppConn plugin 
 #### 1） Download binary package 
@@ -67,7 +67,7 @@ After the exchangis-appconn is installed and deployed, the following steps can b
 2.	 Check whether the project is created synchronously on Exchangis. Successful creation means successful installation of appconn 
 ![image](https://user-images.githubusercontent.com/27387830/169782337-678f2df0-080a-495a-b59f-a98c5a427cf8.png)
 
-For more operation, please refer to [Exchangis 1.0 User Manual](https://github.com/WeBankFinTech/Exchangis/blob/dev-1.0.0-rc/docs/zh_CN/ch1/exchangis_user_manual_cn.md)
+For more operation, please refer to [Exchangis User Manual](docs/zh_CN/ch1/exchangis_user_manual_cn.md)
 
 ### 5.Exchangis AppConn installation principle 
 

docs/en_US/ch1/exchangis_datax_deploy_en.md

@@ -0,0 +1,84 @@
+# DataX engine uses documentation 
+
+### Prepare the environment 
+
+The DataX engine is an indispensable component for executing Exchangis data synchronization tasks. Data synchronization tasks can be performed only after the DataX engine is installed and deployed. Also, ensure that DataX is installed on the deployed machine.
+
+Before you install and deploy DataX engine,  Please complete the installation of Exchangis and related components according to the [Exchangis installation and deployment document](docs/en_US/ch1/exchangis_deploy_en.md), and ensure that the basic functions of the project are available. 
+
+It is strongly recommended that you use the native DataX to perform the test task on this node before performing the DataX task, so as to check whether the environment of this node is normal. 
+
+| Environment variable name | Environment variable content | Remark       |
+| :-----------------------: | :--------------------------: | ------------ |
+|         JAVA_HOME         |    JDK installation path     | Required     |
+|        DATAX_HOME         |   DataX installation path    | Not Required |
+|      DATAX_CONF_DIR       |      DataX config path       | Not Required |
+
+### Prepare installation package 
+
+#### 1）Download binary package 
+
+Exchangis1.1.2 and Linkis 1.4.0 support the mainstream DataX versions 1.4.6 and 1.4.7, and later versions may need to modify some codes for recompilation. 
+
+[Click to jump to Release interface](https://github.com/WeBankFinTech/Exchangis/releases/tag/release-1.1.2)
+
+#### 2）Compile and package 
+
+If you want to develop and compile datax engine yourself, the specific compilation steps are as follows: 
+
+1.clone Exchangis's source code
+
+2.Under exchangis-plugins module, find sqoop engine and compile sqoop separately, as follows :
+
+```
+cd {EXCHANGIS_CODE_HOME}/exchangis-plugins/engine/datax
+mvn clean install
+```
+
+Then the datax engine installation package will be found in this path. 
+
+```
+{EXCHANGIS_CODE_HOME}\exchangis-plugins\datax\target\out\datax
+```
+
+
+### Start deployment
+
+#### 1）DataX engine installation 
+
+1、Get the packed datax.zip material package, the directory structure is
+
+```shell
+datax
+-- dist
+-- plugin
+```
+
+2、Place in the following directory in the linkis installation path
+
+```shell
+cd {LINKIS_HOME}/linkis/lib/linkis-engineconn-plugins
+```
+
+(Note that depending on which users the current datax engine has permissions for, they are generally hadoop user groups and hadoop users)
+
+
+#### 2）Restart linkis-engineplugin service to make datax engine take effect
+
+New engines joining linkis will not take effect until the engineplugin service of linkis is restarted, and the restart script is. /linkis-daemon.sh in the Linkis installation directory. The specific steps are as follows :
+
+```
+cd {LINKIS_INSTALL_HOME}/links/sbin/
+./linkis-daemon.sh restart cg-engineplugin
+```
+
+After the service is successfully started, check whether the datax engine is installed in the linkis database
+
+```shell
+select * from linkis_cg_engine_conn_plugin_bml_resources where engine_conn_type='datax';
+```
+
+At this point, the datax installation and deployment is complete.
+
+For a more detailed introduction of engineplugin, please refer to the following article.   
+https://linkis.apache.org/zh-CN/docs/latest/deployment/install-engineconn