Skip to content

Commit

Permalink
feature support monitoring apache airflow (#1081)
Browse files Browse the repository at this point in the history
  • Loading branch information
luoxuanzao authored Jul 9, 2023
1 parent 1754213 commit f0db2ca
Show file tree
Hide file tree
Showing 2 changed files with 181 additions and 0 deletions.
41 changes: 41 additions & 0 deletions home/docs/help/airflow.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
---
id: airflow
title: 监控:Apache Airflow监控
sidebar_label: Apache Airflow
keywords: [开源监控系统, 开源数据库监控, Apache Airflow监控]
---

> 对Apache Airflow通用性能指标进行采集监控。
### 配置参数

| 参数名称 | 参数帮助描述 |
| ------------ | ------------------------------------------------------------ |
| 监控Host | 被监控的对端IPV4,IPV6或域名。注意⚠️不带协议头(eg: https://, http://) |
| 监控名称 | 标识此监控的名称,名称需要保证唯一性 |
| 端口 | 数据库对外提供的端口,默认为8080 |
| 查询超时时间 | 设置连接未响应的超时时间,单位ms毫秒,默认3000毫秒 |
| HTTPS | 是否启用HTTPS |
| 采集间隔 | 监控周期性采集数据间隔时间,单位秒,可设置的最小间隔为30秒 |
| 是否探测 | 新增监控前是否先探测检查监控可用性,探测成功才会继续新增修改操作 |
| 描述备注 | 更多标识和描述此监控的备注信息,用户可以在这里备注信息 |

### 采集指标

#### 指标集合:airflow_health

| 指标名称 | 指标单位 | 指标帮助描述 |
| ------------ | -------- | -------------------- |
| metadatabase || metadatabase健康情况 |
| scheduler || scheduler健康情况 |
| triggerer || triggerer健康情况 |

#### 指标集合:airflow_version

| 指标名称 | 指标单位 | 指标帮助描述 |
| ----------- | -------- | --------------- |
| value || Airflow版本 |
| git_version || Airflow git版本 |



140 changes: 140 additions & 0 deletions manager/src/main/resources/define/app-airflow.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# The monitoring type category:service-application service monitoring db-database monitoring custom-custom monitoring os-operating system monitoring mid-middleware
# 监控类型所属类别:service-应用服务监控 db-数据库监控 custom-自定义监控 os-操作系统监控 cn-云原生cloud native network-网络监控 mid-中间件
category: custom
# The monitoring: airflow
# 监控类型 airflow
app: airflow
# The monitoring i18n name
# 监控类型国际化名称
name:
zh-CN: Apache Airflow
en-US: Apache Airflow
# 监控所需输入参数定义(根据定义渲染页面UI)
# Input params define for monitoring(render web ui by the definition)
params:
# field-param field key
# field-变量字段标识符
- field: host
# name-param field display i18n name
# name-参数字段显示名称
name:
zh-CN: 服务器Host
en-US: Host
# type-param field type(most mapping the html input type)
# type-字段类型,样式(大部分映射input标签type属性)
type: host
# required-true or false
# required-是否是必输项 true-必填 false-可选
required: true
# required-true or false
# required-是否是必输项 true-必填 false-可选
- field: port
# name-param field display i18n name
# name-参数字段显示名称
name:
zh-CN: 服务端口
en-US: Port
# type-param field type(most mapping the html input type)
# type-字段类型,样式(大部分映射input标签type属性)
type: number
# when type is number, range is required
# 当type为number时,用range表示范围
range: '[0,65535]'
# required-true or false
# required-是否是必输项 true-必填 false-可选
required: true
# required-true or false
# required-是否是必输项 true-必填 false-可选
defaultValue: 8080

- field: ssl
name:
zh-CN: 启用HTTPS
en-US: HTTPS
type: boolean
hide: true
required: true

- field: timeout
name:
zh-CN: 超时时间(ms)
en-US: Timeout(ms)
type: number
hide: true
required: false





# collect metrics config list
# 采集指标组配置列表
metrics:
- name: airflow_health
priority: 0
fields:
- field: metadatabase
type: 1
- field: scheduler
type: 1
- field: triggerer
type: 1
aliasFields:
- $.metadatabase.status
- $.scheduler.status
- $.triggerer.status
calculates:
- metadatabase=$.metadatabase.status
- scheduler=$.scheduler.status
- triggerer=$.triggerer.status
# 监控采集使用协议 eg: sql, ssh, http, telnet, wmi, snmp, sdk
protocol: http
# 当protocol为http协议时具体的采集配置
http:
# 主机host: ipv4 ipv6 域名
host: ^_^host^_^
# 端口
port: ^_^port^_^
# url请求接口路径
url: /api/v1/health
timeout: ^_^timeout^_^
method: GET
parseType: jsonPath
parseScript: '$'

- name: airflow_version
priority: 1
fields:
- field: version
type: 1
- field: git_version
type: 1
# 监控采集使用协议 eg: sql, ssh, http, telnet, wmi, snmp, sdk
protocol: http
# 当protocol为http协议时具体的采集配置
http:
# 主机host: ipv4 ipv6 域名
host: ^_^host^_^
# 端口
port: ^_^port^_^
# url请求接口路径
url: /api/v1/version
timeout: ^_^timeout^_^
method: GET
parseType: jsonPath
parseScript: '$'

0 comments on commit f0db2ca

Please sign in to comment.