Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the support of ISSUE template, Pull request template, and github CI action. #281

Merged
merged 9 commits into from
Apr 29, 2022
103 changes: 103 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
name: Bug report
title: "[Bug] "
description: Problems and issues with code of Exchangis
labels: [bug, triage]
body:
- type: markdown
attributes:
value: |
Thank you for reporting the problem!
Please make sure what you are reporting is a bug with reproducible steps. To ask questions
or share ideas, pleae post on our [Discussion page](https://github.com/WeBankFinTech/Exchangis/discussions) instead.

- type: checkboxes
attributes:
label: Search before asking
description: >
Please make sure to search in the [issues](https://github.com/WeBankFinTech/Exchangis/issues) first to see
whether the same issue was reported already.
options:
- label: >
I searched the [issues](https://github.com/WeBankFinTech/Exchangis/issues) and found no similar
issues.
required: true

- type: dropdown
attributes:
label: Exchangis Component
description: |
What component are you using? Exchangis has many modules, please make sure to choose the module that
you found the bug.
multiple: true
options:
- "exchangis-datasource"
- "exchangis-job-launcher"
- "exchangis-job-server"
- "exchangis-job-builder"
- "exchangis-job-metrics"
- "exchangis-project"
- "exchangis-plugins"
- "exchangis-dao"
- "exchangis-web"
validations:
required: true

- type: textarea
attributes:
label: What happened + What you expected to happen
description: Describe 1. the bug 2. expected behavior 3. useful information (e.g., logs)
placeholder: >
Please provide the context in which the problem occurred and explain what happened. Further,
To Reproduce Steps to reproduce the behavior: 1. Go to '...' 2. Click on '....' 3. Scroll down to '.... 4. See error
please also explain why you think the behaviour is erroneous. It is extremely helpful if you can
copy and paste the fragment of logs showing the exact error messages or wrong behaviour here.

**NOTE**: Expected behavior A clear and concise description of what you expected to happen.Screenshots If applicable, add screenshots to help explain your problem.
validations:
required: true

- type: textarea
attributes:
label: Relevent platform
description: The platform where you occurred this issue
placeholder: >
Please specify Desktop or Smartphone, Version / Dependencies / OS / Browser
validations:
required: true

- type: textarea
attributes:
label: Reproduction script
description: >
Please provide a reproducible script. Providing a narrow reproduction (minimal / no external dependencies) will
help us triage and address issues in the timely manner!
placeholder: >
Please provide a short code snippet (less than 50 lines if possible) that can be copy-pasted to
reproduce the issue. The snippet should have **no external library dependencies**
(i.e., use fake or mock data / environments).

**NOTE**: If the code snippet cannot be run by itself, the issue will be marked as "needs-repro-script"
until the repro instruction is updated.
validations:
required: true

- type: textarea
attributes:
label: Anything else
description: Anything else we need to know?
placeholder: >
How often does this problem occur? (Once? Every time? Only when certain conditions are met?)
Any relevant logs to include? Are there other relevant issues?

- type: checkboxes
attributes:
label: Are you willing to submit a PR?
description: >
This is absolutely not required, but we are happy to guide you in the contribution process
especially if you already have a good understanding of how to implement the fix.
options:
- label: Yes I am willing to submit a PR!

- type: markdown
attributes:
value: "Thanks for completing our form!"
5 changes: 5 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
blank_issues_enabled: fasle
contact_links:
- name: Ask a question or get support
url: https://github.com/WeBankFinTech/Exchangis/discussions
about: Ask a question or request support for using Exchangis
63 changes: 63 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
name: Exchangis feature request
description: Suggest an idea for Exchangis project
title: "[Feature] "
labels: [enhancement]
body:
- type: markdown
attributes:
value: |
Thank you for finding the time to propose a new feature!
We really appreciate the community efforts to improve Exchangis.
- type: checkboxes
attributes:
label: Search before asking
description: >
Please make sure to search in the [issues](https://github.com/WeBankFinTech/Exchangis/issues) first to see
whether the same feature was requested already.
options:
- label: >
I had searched in the [issues](https://github.com/WeBankFinTech/Exchangis/issues) and found no similar
feature requirement.
required: true
- type: textarea
attributes:
label: Problem Description
description: Is your feature request related to a problem? Please describe.

- type: textarea
attributes:
label: Description
description: A short description of your feature

- type: textarea
attributes:
label: Use case
description: >
Describe the use case of your feature request.
placeholder: >
Describe the solution you'd like A clear and concise description of what you want to happen.

- type: textarea
attributes:
label: solutions
description: Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

- type: textarea
attributes:
label: Anything else
description: Anything else we need to know?
placeholder: >
Additional context Add any other context or screenshots about the feature request here.

- type: checkboxes
attributes:
label: Are you willing to submit a PR?
description: >
This is absolutely not required, but we are happy to guide you in the contribution process
especially if you already have a good understanding of how to implement the feature.
options:
- label: Yes I am willing to submit a PR!

- type: markdown
attributes:
value: "Thanks for completing our form!"
28 changes: 28 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
### What is the purpose of the change
(For example: Exchangis-Job defines the core ability of Exchangis, it provides the abilities of job management, job transform, and job launch.
Related issues: #50. )

### Brief change log
(for example:)
- defines the job server module of Exchangis;
- defines the job launcher module of Exchangis;
- defines the job metrics module of Exchangis.

### Verifying this change
(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)
- Added tests for creating and execute the Exchangis jobs and verify the availability of different Exchangis Job, such as sqoop job, datax job.

### Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (yes / no)
- Anything that affects deployment: (yes / no / don't know)
- The Core framework, i.e., JobManager, Server.: (yes / no)

### Documentation
- Does this pull request introduce a new feature? (yes / no)
- If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
55 changes: 55 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
#
# Copyright 2019 WeBank.
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

name: Exchangis CI Actions

on:
push:
pull_request:

jobs:
build:

runs-on: ubuntu-latest

strategy:
matrix:
node-version: [14.17.3]
# See supported Node.js release schedule at https://nodejs.org/en/about/releases/

steps:
- name: Checkout
uses: actions/checkout@v2
- name: Set up JDK 8
uses: actions/setup-java@v2
with:
distribution: 'adopt'
java-version: 8
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v2
with:
node-version: ${{ matrix.node-version }}
- name: Build backend by maven
run: |
mvn -N install
mvn clean package
- name: Build frontend by node.js
run: |
cd web
npm install
npm run build
48 changes: 48 additions & 0 deletions .github/workflows/check_license.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
#
# Copyright 2019 WeBank.
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#

name: Exchangis License check

on: [push, pull_request]

jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout source
uses: actions/checkout@v2
- name: Set up JDK 8
uses: actions/setup-java@v2
with:
java-version: '8'
distribution: 'adopt'
- name: mvn -N install
run:
mvn -N install
- name: License check with Maven
run: |
rat_file=`mvn apache-rat:check | { grep -oe "\\S\\+/rat.txt" || true; }`
echo "rat_file=$rat_file"
if [[ -n "$rat_file" ]];then echo "check error!" && cat $rat_file && exit 123;else echo "check success!" ;fi
- name: Upload the report
uses: actions/upload-artifact@v2
with:
name: license-check-report
path: "**/target/rat.txt"
68 changes: 68 additions & 0 deletions README-ZH.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
[![License](https://img.shields.io/badge/license-Apache%202-4EB1BA.svg)](https://www.apache.org/licenses/LICENSE-2.0.html)

[English](README.md) | 中文

## 项目简介

Exchangis 是微众银行大数据平台 WeDataSphere 自研的数据交换工具,支持异构数据源之间的结构化和非结构化数据传输同步。

Exchangis 抽象了一套统一的数据源和同步作业定义插件,允许用户快速接入新的数据源,并只需在数据库中简单配置即可在页面中使用。

基于插件化的框架设计,及计算中间件 [Linkis](https://github.com/apache/incubator-linkis),Exchangis 可快速集成对接 Linkis 已集成的数据同步引擎,将 Exchangis 的同步作业转换成 Linkis 数据同步引擎的数据同步作业。

借助于 [Linkis](https://github.com/apache/incubator-linkis) 计算中间件的连接、复用与简化能力,Exchangis 天生便具备了高并发、高可用、多租户隔离和资源管控的金融级数据同步能力。

## 核心特点

### 1. 数据源管理

基于 Linkis DataSource,抽象了底层数据源在 Exchangis 作为一个同步作业的 Source 和 Sink 所必须的所有能力。

- **多传输引擎支持**
传输引擎可横向扩展;
当前版本完整聚合了离线批量引擎DataX、部分聚合了大数据批量导数引擎SQOOP

- **近实时任务管控**
快速抓取传输任务日志以及传输速率等信息,实时关闭任务;
可根据带宽状况对任务进行动态限流

- **支持无结构化传输**
DataX框架改造,单独构建二进制流快速通道,适用于无数据转换的纯数据同步场景。

- **任务状态自检**
监控长时间运行的任务和状态异常任务,及时释放占用的资源并发出告警。

## 与现有的系统的对比
对现有的一些数据交换工具和平台的对比:

| 功能模组 | 描述 | Exchangis | DataX | Sqoop | DataLink | DBus |
| :----: | :----: |-------|-------|-------|-------|-------|
| UI | 集成便捷的管理界面和监控窗口| 已集成 | 无 | 无 | 已集成 |已集成 |
| 安装部署 | 部署难易程度和第三方依赖 | 一键部署,无依赖 | 无依赖 | 依赖Hadoop环境 | 依赖Zookeeper | 依赖大量第三方组件 |
| 数据权限管理| 多租户权限配置和数据源权限管控 | 支持 | 不支持 | 不支持 | 不支持 | 支持 |
| |动态限流传输 | 支持 | 部分支持,无法动态调整 | 部分支持,无法动态调整| 支持 | 支持,借助Kafka |
| 数据传输| 无结构数据二进制传输 | 支持,快速通道 | 不支持 | 不支持 | 不支持,都是记录 | 不支持,需要转化为统一消息格式|
| | 嵌入处理代码 | 支持,动态编译 | 不支持 | 不支持 | 不支持 | 部分支持 |
| | 传输断点恢复 | 支持(未开源) | 不支持,只能重试 | 不支持,只能重试 | 支持 | 支持 |
| 服务高可用 | 服务多点,故障不影响使用| 应用高可用,传输单点(分布式架构规划中) | 单点服务(开源版本) | 传输多点 | 应用、传输高可用 | 应用、传输高可用 |
| 系统管理 | 节点、资源管理 | 支持 | 不支持 | 不支持 | 支持 | 支持 |

## 整体设计

### 架构设计

![架构设计](../../../images/zh_CN/ch1/architecture.png)

## 相关文档
[安装部署文档](exchangis_deploy_cn.md)
[用户手册](exchangis_user_manual_cn.md)

## 交流贡献

如果您想得到最快的响应,请给我们提 issue,或者扫码进群:

![communication](../../../images/communication.png)

## License

Exchangis is under the Apache 2.0 License. See the [License](../../../LICENSE) file for details.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[![License](https://img.shields.io/badge/license-Apache%202-4EB1BA.svg)](https://www.apache.org/licenses/LICENSE-2.0.html)

English | [中文](docs/zh_CN/ch1/README.md)
English | [中文](README-ZH.md)

## Introduction
Exchangis is a lightweight,highly extensible data exchange platform that supports data transmission between structured and unstructured heterogeneous data sources. On the application layer, it has business features such as data permission management and control, high availability of node services and multi-tenant resource isolation. On the data layer, it also has architectural characteristics such as diversified transmission architecture, module plug-in and low coupling of components.
Expand Down