Branch 2.0 var merge #27422

eldenmoon · 2023-11-22T10:27:13Z

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

In the past, two conditions needed to be met when sampling a partitioned table: 1. Data is evenly distributed between partitions; 2. Data is evenly distributed between buckets. Finally, the number of sampled rows in each partition and each bucket is the same. Now, sampling will be proportional to the number of partitioned and bucketed rows.

…pache#26414)

)

apache#26147 (apache#26417)

…pache#26116 (apache#26419)

… check apache#25594 (apache#26422)

…age apache#25552 (apache#26423) Remove unless code and add call method for coverage

…pache#26443) backport apache#26240

…#26441)

…ache#26401 (apache#26472)

… topn in some condition apache#26473 (apache#26481)

…pache#25652 (apache#26430)

…che#25709) (apache#26305) 1. generalized debug point facilities from docker suites for fault-injection/stubbing cases 2. add segcompaction fault-injection cases for demonstration 3. add -238 TOO_MANY_SEGMENTS fault-injection case for good Co-authored-by: zhengyu <freeman.zhang1992@gmail.com>

…on (apache#26516) row numbers miscounting because of race condition, will cause load to fail sometimes with warning 'the rows number written doesn't match'. Signed-off-by: freemandealer <freeman.zhang1992@gmail.com>

…) (apache#26547)

…pache#26326) (apache#26530) we use a map to record rewrite cte children result to avoid rewrite twice in cost based rewritter. However, we record cte outer and inner in one map, and use null as outer result's key, use cte id as inner result's key. This is wrong, because every anchor has an outer, and we could only record one outer. So when we use the cache in cost based rewritter, we get wrong outer plan from the cache. Then the error will be thrown as below: ``` Caused by: java.lang.IllegalArgumentException: Stats for CTE: CTEId#1 not found at com.google.common.base.Preconditions.checkArgument(Preconditions.java:143) ~[guava-32.1.2-jre.jar:?] at org.apache.doris.nereids.stats.StatsCalculator.visitLogicalCTEConsumer(StatsCalculator.java:1049) ~[classes/:?] at org.apache.doris.nereids.stats.StatsCalculator.visitLogicalCTEConsumer(StatsCalculator.java:147) ~[classes/:?] at org.apache.doris.nereids.trees.plans.logical.LogicalCTEConsumer.accept(LogicalCTEConsumer.java:111) ~[classes/:?] at org.apache.doris.nereids.stats.StatsCalculator.estimate(StatsCalculator.java:222) ~[classes/:?] at org.apache.doris.nereids.stats.StatsCalculator.estimate(StatsCalculator.java:200) ~[classes/:?] at org.apache.doris.nereids.jobs.cascades.DeriveStatsJob.execute(DeriveStatsJob.java:108) ~[classes/:?] at org.apache.doris.nereids.jobs.scheduler.SimpleJobScheduler.executeJobPool(SimpleJobScheduler.java:39) ~[classes/:?] at org.apache.doris.nereids.jobs.executor.Optimizer.execute(Optimizer.java:51) ~[classes/:?] at org.apache.doris.nereids.jobs.rewrite.CostBasedRewriteJob.getCost(CostBasedRewriteJob.java:98) ~[classes/:?] at org.apache.doris.nereids.jobs.rewrite.CostBasedRewriteJob.execute(CostBasedRewriteJob.java:64) ~[classes/:?] at org.apache.doris.nereids.jobs.executor.AbstractBatchJobExecutor.execute(AbstractBatchJobExecutor.java:119) ~[classes/:?] at org.apache.doris.nereids.rules.rewrite.RewriteCteChildren.visit(RewriteCteChildren.java:72) ~[classes/:?] at org.apache.doris.nereids.rules.rewrite.RewriteCteChildren.visit(RewriteCteChildren.java:56) ~[classes/:?] at org.apache.doris.nereids.trees.plans.visitor.PlanVisitor.visitLogicalSink(PlanVisitor.java:118) ~[classes/:?] at org.apache.doris.nereids.trees.plans.visitor.SinkVisitor.visitLogicalResultSink(SinkVisitor.java:72) ~[classes/:?] at org.apache.doris.nereids.trees.plans.logical.LogicalResultSink.accept(LogicalResultSink.java:58) ~[classes/:?] at org.apache.doris.nereids.rules.rewrite.RewriteCteChildren.visitLogicalCTEAnchor(RewriteCteChildren.java:86) ~[classes/:?] at org.apache.doris.nereids.rules.rewrite.RewriteCteChildren.visitLogicalCTEAnchor(RewriteCteChildren.java:56) ~[classes/:?] at org.apache.doris.nereids.trees.plans.logical.LogicalCTEAnchor.accept(LogicalCTEAnchor.java:60) ~[classes/:?] at org.apache.doris.nereids.rules.rewrite.RewriteCteChildren.visitLogicalCTEAnchor(RewriteCteChildren.java:86) ~[classes/:?] at org.apache.doris.nereids.rules.rewrite.RewriteCteChildren.visitLogicalCTEAnchor(RewriteCteChildren.java:56) ~[classes/:?] at org.apache.doris.nereids.trees.plans.logical.LogicalCTEAnchor.accept(LogicalCTEAnchor.java:60) ~[classes/:?] at org.apache.doris.nereids.rules.rewrite.RewriteCteChildren.rewriteRoot(RewriteCteChildren.java:67) ~[classes/:?] at org.apache.doris.nereids.jobs.rewrite.CustomRewriteJob.execute(CustomRewriteJob.java:58) ~[classes/:?] at org.apache.doris.nereids.jobs.executor.AbstractBatchJobExecutor.execute(AbstractBatchJobExecutor.java:119) ~[classes/:?] at org.apache.doris.nereids.NereidsPlanner.rewrite(NereidsPlanner.java:275) ~[classes/:?] at org.apache.doris.nereids.NereidsPlanner.plan(NereidsPlanner.java:218) ~[classes/:?] at org.apache.doris.nereids.NereidsPlanner.plan(NereidsPlanner.java:118) ~[classes/:?] at org.apache.doris.nereids.trees.plans.commands.ExplainCommand.run(ExplainCommand.java:81) ~[classes/:?] at org.apache.doris.qe.StmtExecutor.executeByNereids(StmtExecutor.java:550) ~[classes/:?] ```

) (apache#26531) pick from master PR: apache#26330 commit id: a89477e ExpressionDeepCopier not process VirtualReference, so we generate inline plan with mistake.

…ons (apache#26444) (apache#26568)

…directly (apache#26456) (apache#26583) Signed-off-by: Jack Drogon <jack.xsuperman@gmail.com>

…scanner apache#24199 (apache#25547)" (apache#26613) This reverts commit 9a19581 to investigate ANALYZE TABLE WITH SYNC problem

…ner. (apache#26542)

(apache#26533)

…ng cleartext passwords in config (apache#26278) (apache#26532) Doris FE will check if stream load http request has auth token after checking password failed; Plugin audit-log loader can use auth token if plugin config set use_auth_token to true Co-authored-by: Mingyu Chen <morningman.cmy@gmail.com>

… have back quote when querying the JDBC appearance (apache#26479) (apache#26560) master pr: apache#26479

…pare a forward master sql (apache#26512) (apache#26638)

…pache#26121 (apache#26376)

…he#27335

…ithout data (apache#27276)

…-2.0-var

github-actions · 2023-11-22T10:28:11Z

`sh-checker report`

To get the full details, please check in the job output.

shellcheck errors


'shellcheck ' returned error 1 finding the following syntactical issues:

----------

In pytest/deploy/qe/init_data.sh line 23:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data.sh line 27:
curl --location-trusted -T baseall.txt -u "${FE_USER}":"${FE_PASSWORD}" http://"${FE_HOST}":"${FE_WEB_PORT}"/api/"${FE_DB}"/baseall/_stream_load
                                                                                             ^------------^ SC2154 (warning): FE_WEB_PORT is referenced but not assigned.


In pytest/deploy/qe/init_data_broker.sh line 24:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data_broker.sh line 33:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_1 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE baseall COLUMNS TERMINATED BY '${separator}') ${broker_property}"
                                                                                                                           ^---------------^ SC2154 (warning): hdfs_data_path is referenced but not assigned.
                                                                                                                                                                                                                       ^----------------^ SC2154 (warning): broker_property is referenced but not assigned.


In pytest/deploy/qe/init_data_duplicate.sh line 23:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data_duplicate.sh line 32:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_1 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE baseall COLUMNS TERMINATED BY '${separator}') ${broker_property}"
                                                                                                                           ^---------------^ SC2154 (warning): hdfs_data_path is referenced but not assigned.
                                                                                                                                                                                                                       ^----------------^ SC2154 (warning): broker_property is referenced but not assigned.


In pytest/deploy/qe/init_data_list.sh line 23:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data_list.sh line 32:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_1 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE baseall COLUMNS TERMINATED BY '${separator}') ${broker_property}"
                                                                                                                           ^---------------^ SC2154 (warning): hdfs_data_path is referenced but not assigned.
                                                                                                                                                                                                                       ^----------------^ SC2154 (warning): broker_property is referenced but not assigned.


In pytest/deploy/qe/init_data_multi.sh line 23:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data_multi.sh line 32:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_1 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE baseall COLUMNS TERMINATED BY '${separator}') ${broker_property}"
                                                                                                                           ^---------------^ SC2154 (warning): hdfs_data_path is referenced but not assigned.
                                                                                                                                                                                                                       ^----------------^ SC2154 (warning): broker_property is referenced but not assigned.


In pytest/deploy/qe/init_data_stream.sh line 21:
cd ../../hdfs/data/qe/
^--------------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
cd ../../hdfs/data/qe/ || exit


In pytest/deploy/qe/init_data_stream.sh line 23:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data_stream.sh line 27:
curl --location-trusted -T baseall.txt -u "${FE_USER}":"${FE_PASSWORD}" http://"${FE_HOST}":"${FE_WEB_PORT}"/api/"${FE_DB}"/baseall/_stream_load
                                                                                             ^------------^ SC2154 (warning): FE_WEB_PORT is referenced but not assigned.


In pytest/deploy/qe/init_data_unique.sh line 23:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data_unique.sh line 32:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_1 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE baseall COLUMNS TERMINATED BY '${separator}') ${broker_property}"
                                                                                                                           ^---------------^ SC2154 (warning): hdfs_data_path is referenced but not assigned.
                                                                                                                                                                                                                       ^----------------^ SC2154 (warning): broker_property is referenced but not assigned.


In pytest/deploy/qe/init_ext.sh line 19:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e 'DROP DATABASE IF EXISTS test_query_qa'
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.


In pytest/deploy/qe/init_mysql.sh line 20:
mysql -h "${MYSQL_HOST}" -P"${MYSQL_PORT}" -u"${MYSQL_USER}" -p"${MYSQL_PASSWORD}" -e "DROP DATABASE IF EXISTS ${MYSQL_DB}"
          ^-----------^ SC2154 (warning): MYSQL_HOST is referenced but not assigned.
                            ^-----------^ SC2154 (warning): MYSQL_PORT is referenced but not assigned.
                                              ^-----------^ SC2154 (warning): MYSQL_USER is referenced but not assigned.
                                                                ^---------------^ SC2154 (warning): MYSQL_PASSWORD is referenced but not assigned.
                                                                                                               ^---------^ SC2154 (warning): MYSQL_DB is referenced but not assigned.


In pytest/deploy/qe/init_mysql.sh line 24:
mysql -h "${MYSQL_HOST}" -P"${MYSQL_PORT}" -u"${MYSQL_USER}" -p"${MYSQL_PASSWORD}" "${MYSQL_DB}" -e 'CREATE TABLE `baseall` ( `k1` tinyint(4) DEFAULT NULL, `k2` smallint(6) DEFAULT NULL, `k3` int(11) DEFAULT NULL, `k4` bigint(20) DEFAULT NULL, `k5` decimal(9,3) DEFAULT NULL, `k6` char(5) DEFAULT NULL, `k10` date DEFAULT NULL, `k11` datetime DEFAULT NULL, `k7` varchar(20) DEFAULT NULL, `k8` double DEFAULT NULL, `k9` float DEFAULT NULL)'
                                                                                                    ^-- SC2016 (info): Expressions don't expand in single quotes, use double quotes for that.


In pytest/deploy/qe/init_mysql.sh line 25:
mysql -h "${MYSQL_HOST}" -P"${MYSQL_PORT}" -u"${MYSQL_USER}" -p"${MYSQL_PASSWORD}" "${MYSQL_DB}" -e 'CREATE TABLE `bigtable` ( `k1` tinyint(4) DEFAULT NULL, `k2` smallint(6) DEFAULT NULL, `k3` int(11) DEFAULT NULL, `k4` bigint(20) DEFAULT NULL, `k5` decimal(9,3) DEFAULT NULL, `k6` char(5) DEFAULT NULL, `k10` date DEFAULT NULL, `k11` datetime DEFAULT NULL, `k7` varchar(20) DEFAULT NULL, `k8` double DEFAULT NULL, `k9` float DEFAULT NULL)'
                                                                                                    ^-- SC2016 (info): Expressions don't expand in single quotes, use double quotes for that.


In pytest/deploy/qe/init_mysql.sh line 26:
mysql -h "${MYSQL_HOST}" -P"${MYSQL_PORT}" -u"${MYSQL_USER}" -p"${MYSQL_PASSWORD}" "${MYSQL_DB}" -e 'CREATE TABLE `test` ( `k1` tinyint(4) DEFAULT NULL, `k2` smallint(6) DEFAULT NULL, `k3` int(11) DEFAULT NULL, `k4` bigint(20) DEFAULT NULL, `k5` decimal(9,3) DEFAULT NULL, `k6` char(5) DEFAULT NULL, `k10` date DEFAULT NULL, `k11` datetime DEFAULT NULL, `k7` varchar(20) DEFAULT NULL, `k8` double DEFAULT NULL, `k9` float DEFAULT NULL)'
                                                                                                    ^-- SC2016 (info): Expressions don't expand in single quotes, use double quotes for that.


In pytest/qe/palo2/data/init_lateral_view.sh line 19:
export hdfs_path=${hdfs_data_path}/qe/lateral_view
                 ^---------------^ SC2154 (warning): hdfs_data_path is referenced but not assigned.


In pytest/qe/palo2/data/init_lateral_view.sh line 21:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS test_lateral_view"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.


In pytest/qe/palo2/data/init_lateral_view.sh line 25:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL test_lateral_view.label_1 (DATA INFILE('${hdfs_path}/lateral_view_data') INTO TABLE lateral_view_data (k1,k2,k3,k4,k5,k6,k7,k8) set (k9=bitmap_from_string(k8))) ${broker_property}"
                                                                                                                                                                                                                                                             ^----------------^ SC2154 (warning): broker_property is referenced but not assigned.


In pytest/run_case.sh line 19:
export ROOT_PATH=$(pwd)
       ^-------^ SC2155 (warning): Declare and assign separately to avoid masking return values.


In pytest/run_case.sh line 26:
rm -f ${ROOT_PATH}/*.xml | tee
      ^----------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
rm -f "${ROOT_PATH}"/*.xml | tee


In pytest/run_case.sh line 27:
rm -f ${ROOT_PATH}/*.html | tee
      ^----------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
rm -f "${ROOT_PATH}"/*.html | tee


In pytest/run_case.sh line 28:
rm -rf ${REPORT_PATH}
       ^------------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
rm -rf "${REPORT_PATH}"


In pytest/run_case.sh line 29:
mkdir -p ${REPORT_PATH}
         ^------------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
mkdir -p "${REPORT_PATH}"


In pytest/run_case.sh line 31:
cd ${SYS_PATH}
^------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
   ^---------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
cd "${SYS_PATH}" || exit


In pytest/run_case.sh line 57:
    cd ${ENV_PATH}
    ^------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
       ^---------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
    cd "${ENV_PATH}" || exit


In pytest/run_case.sh line 67:
    ls ${case_file}
       ^----------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
    ls "${case_file}"


In pytest/run_case.sh line 94:
    cd ${SYS_PATH}
    ^------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
       ^---------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
    cd "${SYS_PATH}" || exit


In pytest/run_case.sh line 131:
    cd "${QE_PATH}"
    ^-------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${QE_PATH}" || exit


In pytest/run_case.sh line 136:
    cd "${QE_BUG_PATH}"
    ^-----------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${QE_BUG_PATH}" || exit


In pytest/run_case.sh line 193:
    cd "${SYS_PATH}"
    ^--------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}" || exit


In pytest/run_case.sh line 200:
    cd "${SYS_PATH}"/test_sys_routine_load/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_routine_load/ || exit


In pytest/run_case.sh line 207:
    cd "${SYS_PATH}"/test_sys_broker_load/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_broker_load/ || exit


In pytest/run_case.sh line 214:
    cd "${SYS_PATH}"/test_sys_backup_restore
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_backup_restore || exit


In pytest/run_case.sh line 224:
    cd "${SYS_PATH}"/test_sys_stream_load/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_stream_load/ || exit


In pytest/run_case.sh line 241:
    cd "${SYS_PATH}"/test_sys_alter_for_uniq/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_alter_for_uniq/ || exit


In pytest/run_case.sh line 258:
    cd "${SYS_PATH}"/test_sys_alter_for_duplicate/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_alter_for_duplicate/ || exit


In pytest/run_case.sh line 275:
    cd "${SYS_PATH}"/test_sys_alter_for_aggregate/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_alter_for_aggregate/ || exit


In pytest/run_case.sh line 283:
    cd "${SYS_PATH}"/test_sys_bitmap/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_bitmap/ || exit


In pytest/run_case.sh line 291:
    cd "${SYS_PATH}"/test_sys_export/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_export/ || exit


In pytest/run_case.sh line 296:
    cd "${SYS_PATH}"/test_sys_string
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_string || exit


In pytest/run_case.sh line 304:
    cd "${SYS_PATH}"/test_sys_array
    ^-----------------------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_array || exit


In pytest/run_restart_case.sh line 19:
export ROOT_PATH=$(pwd)
       ^-------^ SC2155 (warning): Declare and assign separately to avoid masking return values.


In pytest/run_restart_case.sh line 42:
cd "${SYS_PATH}"/test_scene
^-------------------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
cd "${SYS_PATH}"/test_scene || exit


In pytest/run_restart_case.sh line 48:
    cd "${SYS_PATH}"
    ^--------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}" || exit


In pytest/run_restart_case.sh line 55:
cd "${SYS_PATH}"
^--------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
cd "${SYS_PATH}" || exit

For more information:
  https://www.shellcheck.net/wiki/SC2154 -- FE_DB is referenced but not assig...
  https://www.shellcheck.net/wiki/SC2155 -- Declare and assign separately to ...
  https://www.shellcheck.net/wiki/SC2164 -- Use 'cd ... || exit' or 'cd ... |...
----------

You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
  # shellcheck disable=NNNN
above the line that contains the issue, where NNNN is the error code;
3. Add '-e NNNN' to the SHELLCHECK_OPTS setting in your .yml action file.

shfmt errors


'shfmt ' returned error 1 finding the following formatting issues:

----------
--- pytest/deploy/qe/init_data.sh.orig
+++ pytest/deploy/qe/init_data.sh
@@ -1,4 +1,4 @@
-#!/bin/sh	
+#!/bin/sh
 # Licensed to the Apache Software Foundation (ASF) under one
 # or more contributor license agreements.  See the NOTICE file
 # distributed with this work for additional information
@@ -32,4 +32,3 @@
 
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" "${FE_DB}" -e 'create table bigtable(k1 tinyint, k2 smallint, k3 int, k4 bigint, k5 decimal(9,3), k6 char(5), k10 date, k11 datetime, k7 varchar(20), k8 double max, k9 float sum) engine=olap distributed by hash(k1) buckets 5 properties("storage_type"="column")'
 curl --location-trusted -T baseall.txt -u "${FE_USER}":"${FE_PASSWORD}" http://"${FE_HOST}":"${FE_WEB_PORT}"/api/"${FE_DB}"/bigtable/_stream_load
-
--- pytest/deploy/qe/init_data_broker.sh.orig
+++ pytest/deploy/qe/init_data_broker.sh
@@ -20,7 +20,6 @@
 # hdfs_data_path='hdfs://xxxxxx'
 # broker_property="WITH BROKER 'hdfs' ('username'='root', 'password'='')"
 
-
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "create database ${FE_DB}"
 
@@ -29,7 +28,6 @@
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "create table ${FE_DB}.bigtable(k1 tinyint, k2 smallint, k3 int, k4 bigint, k5 decimal(9,3), k6 char(5), k10 date, k11 datetime, k7 varchar(20), k8 double max, k9 float sum) engine=olap distributed by hash(k1) buckets 5 properties('storage_type'='column')"
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "CREATE TABLE ${FE_DB}.all_type_table (tinyint_key tinyint(4) NULL, smallint_key smallint(6) NULL, int_key int(11) NOT NULL, bigint_key bigint(20) NULL, char_50_key char(50) NULL, character_key varchar(500) NOT NULL, char_key char(1) NULL, character_most_key varchar(65533) NULL, decimal_key decimal(20, 6) NULL, decimal_most_key decimal(27, 9) NULL, date_key date NULL, datetime_key datetime NULL, tinyint_value tinyint(4) SUM NULL, smallint_value smallint(6) SUM NULL, int_value int(11) SUM NULL, largeint_value largeint SUM NULL, char_50_value char(50) REPLACE NULL, character_value varchar(500) REPLACE NULL, char_value char(1) REPLACE NULL, character_most_value varchar(65533) REPLACE NULL, decimal_value decimal(20, 6) SUM NULL, decimal_most_value decimal(27, 9) SUM NULL, date_value_replace date REPLACE NULL, date_value_max date REPLACE NULL, date_value_min date REPLACE NULL, datetime_value_replace datetime REPLACE NULL, datetime_value_max datetime REPLACE NULL, datetime_value_min datetime REPLACE NULL, float_value float SUM NULL, double_value double SUM NULL) ENGINE=OLAP DISTRIBUTED BY HASH(tinyint_key) BUCKETS 5 PROPERTIES ('storage_type' = 'COLUMN')"
 
-
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_1 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE baseall COLUMNS TERMINATED BY '${separator}') ${broker_property}"
 
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_2 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE test COLUMNS TERMINATED BY '${separator}') ${broker_property}"
--- pytest/deploy/qe/init_data_list.sh.orig
+++ pytest/deploy/qe/init_data_list.sh
@@ -37,5 +37,4 @@
 
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_4 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE bigtable COLUMNS TERMINATED BY '${separator}') ${broker_property}"
 
-
 sleep 20
--- pytest/deploy/qe/init_data_stream.sh.orig
+++ pytest/deploy/qe/init_data_stream.sh
@@ -1,4 +1,4 @@
-#!/bin/sh	
+#!/bin/sh
 # Licensed to the Apache Software Foundation (ASF) under one
 # or more contributor license agreements.  See the NOTICE file
 # distributed with this work for additional information
@@ -32,4 +32,3 @@
 
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" "${FE_DB}" -e 'create table bigtable(k1 tinyint, k2 smallint, k3 int, k4 bigint, k5 decimal(9,3), k6 char(5), k10 date, k11 datetime, k7 varchar(20), k8 double max, k9 float sum) engine=olap distributed by hash(k1) buckets 5 properties("storage_type"="column")'
 curl --location-trusted -T baseall.txt -u "${FE_USER}":"${FE_PASSWORD}" http://"${FE_HOST}":"${FE_WEB_PORT}"/api/"${FE_DB}"/bigtable/_stream_load
-
--- pytest/download_data.sh.orig
+++ pytest/download_data.sh
@@ -16,11 +16,13 @@
 # specific language governing permissions and limitations
 # under the License.
 
-
 set -eo pipefail
 
 PYTEST_HOME=$(dirname "$0")
-PYTEST_HOME=$(cd "${PYTEST_HOME}"; pwd)
+PYTEST_HOME=$(
+    cd "${PYTEST_HOME}"
+    pwd
+)
 
 export PYTEST_HOME
 
@@ -39,7 +41,7 @@
     wget -O "${HDFS_DATA}" https://palo-qa.cdn.bcebos.com/pytest_data/hdfs_data.tar.gz
     rm -rf hdfs
     mkdir -p "${PYTEST_HOME}"/hdfs/
-    tar xzf "${HDFS_DATA}" -C "${PYTEST_HOME}"/hdfs/ 
+    tar xzf "${HDFS_DATA}" -C "${PYTEST_HOME}"/hdfs/
     echo "hdfs data download finished, need to put to hdfs"
 fi
 
@@ -69,6 +71,3 @@
     fi
     echo "local data download finished"
 fi
-
-
-
--- pytest/qe/palo2/data/init_lateral_view.sh.orig
+++ pytest/qe/palo2/data/init_lateral_view.sh
@@ -74,7 +74,7 @@
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e 'create table test_lateral_view.test_multi_partition_bucket(k1 int, k2 int, k3 int, k4 varchar(500), k5 varchar(500), k6 varchar(500), k7 varchar(500), k8 varchar(500), k9 bitmap bitmap_union) PARTITION BY RANGE(k1,k2) (partition p1 values less than ("-32768","-8"), partition p2 values less than ("0","0"),partition p3 values less than ("32768","8"),partition p4 values less than ("65536","18")) DISTRIBUTED BY HASH(k1,k2,k3,k4,k5,k6,k7,k8)'
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL test_lateral_view.label_18 (DATA INFILE('${hdfs_path}/lateral_view_data') INTO TABLE test_multi_partition_bucket (k1,k2,k3,k4,k5,k6,k7,k8) set (k9=bitmap_from_string(k8))) ${broker_property}"
 
-mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e  'create table test_lateral_view.test_explode_split_type(k1 int, k2 char(255), k3 varchar(500), k4 string) DISTRIBUTED BY HASH(k1)'
+mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e 'create table test_lateral_view.test_explode_split_type(k1 int, k2 char(255), k3 varchar(500), k4 string) DISTRIBUTED BY HASH(k1)'
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL test_lateral_view.label_19 (DATA INFILE('${hdfs_path}/lateral_view_data') INTO TABLE test_explode_split_type (k1,v2,v3,v4,v5,v6,v7,v8) set (k2=v4,k3=v4,k4=v4)) ${broker_property}"
 
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e 'create table test_lateral_view.test_explode_split_type_check(k1 int, k2 varchar(50)) DISTRIBUTED BY HASH(k1)'
@@ -85,4 +85,3 @@
 
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e 'create table test_lateral_view.lateral_view_multi_check(k1 int, k2 int, k3 int, k4 varchar(200), k5 bigint, k6 bigint, k7 double) DISTRIBUTED BY HASH(k1)'
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL test_lateral_view.label_22 (DATA INFILE('${hdfs_path}/lateral_view_multi_check') INTO TABLE lateral_view_multi_check (k1,k2,k3,k4,k5,k6,k7)) ${broker_property}"
-
--- pytest/run_case.sh.orig
+++ pytest/run_case.sh
@@ -46,13 +46,11 @@
 exec 3<>/tmp/fd1
 rm -rf /tmp/fd1
 
-for ((i=1;i<=PROCESS_NUM;i++))
-do
-        echo >&3
+for ((i = 1; i <= PROCESS_NUM; i++)); do
+    echo >&3
 done
 
-function check_cluster ()
-{
+function check_cluster() {
     # check if fe or be dead and start it
     cd ${ENV_PATH}
     python node_op.py -c '''
@@ -61,42 +59,37 @@
     '''
 }
 
-function pytest_execute ()
-{
+function pytest_execute() {
     case_file=$1
     ls ${case_file}
-    pytest -sv --junit-xml="${REPORT_PATH}"/"${case_file%.py}".xml --html="${REPORT_PATH}"/"${case_file%.py}".html  "${case_file}" --tb=native 2>&1 | tee "${case_file%.py}".log
+    pytest -sv --junit-xml="${REPORT_PATH}"/"${case_file%.py}".xml --html="${REPORT_PATH}"/"${case_file%.py}".html "${case_file}" --tb=native 2>&1 | tee "${case_file%.py}".log
     sleep 1
 }
 
-function case_execute ()
-{
-    for case_file in "$@"
-    do
+function case_execute() {
+    for case_file in "$@"; do
         read -r -u3
         {
             echo "run ${case_file}"
             pytest_execute "${case_file}"
             if [[ "${RESTART_BE_AUTO}" == true ]]; then
                 check_cluster
-            fi     
+            fi
             echo >&3
-        }&
+        } &
     done
 
 }
 
 # 执行时间较长，最先执行
-function long_time_case ()
-{
+function long_time_case() {
     case_files='test_sys_partition_complex_without_restart_be.py 
                 test_sys_storage_medium.py test_sys_delete_load.py'
     cd ${SYS_PATH}
     case_execute "${case_files}"
 }
 
-function query_case ()
-{
+function query_case() {
     # query case
     case_files='test_query_predicates.py 
                 test_query_others.py
@@ -137,8 +130,7 @@
     case_execute "${case_files}"
 }
 
-function sys_case ()
-{
+function sys_case() {
     # sys case
     case_files='test_sys_verify.py
                 test_sys_bloom_filter_a.py
@@ -290,7 +282,7 @@
                 test_select_into_query.py'
     cd "${SYS_PATH}"/test_sys_export/
     case_execute "${case_files}"
-   
+
     # sys/test_sys_string
     case_files='test_sys_string_basic.py'
     cd "${SYS_PATH}"/test_sys_string
--- pytest/run_restart_case.sh.orig
+++ pytest/run_restart_case.sh
@@ -29,11 +29,10 @@
     testsuite=normal
 fi
 
-function pytest_execute()
-{
+function pytest_execute() {
     case_file=$1
     ls "${case_file}"
-    pytest -sv --junit-xml="${REPORT_PATH}"/"${case_file%.py}".xml --html="${REPORT_PATH}"/"${case_file%.py}".html  "${case_file}" --tb=native 2>&1 | tee "${case_file%.py}".log
+    pytest -sv --junit-xml="${REPORT_PATH}"/"${case_file%.py}".xml --html="${REPORT_PATH}"/"${case_file%.py}".html "${case_file}" --tb=native 2>&1 | tee "${case_file%.py}".log
     sleep 1
 }
 
@@ -43,8 +42,7 @@
 pytest_execute "${case}"
 
 cases='test_sys_update_restart.py test_sys_partition_complex_with_restart_be.py test_sys_restart.py test_sys_binlog_restart.py test_sys_resource_tag.py'
-for i in ${cases}
-do
+for i in ${cases}; do
     cd "${SYS_PATH}"
     echo "${i}"
     pytest_execute "${i}"
----------

You can reformat the above files to meet shfmt's requirements by typing:

  shfmt  -w filename

eldenmoon · 2023-11-22T10:30:52Z

run buildall

github-actions · 2023-11-22T10:31:41Z

`sh-checker report`

To get the full details, please check in the job output.

shellcheck errors


'shellcheck ' returned error 1 finding the following syntactical issues:

----------

In pytest/deploy/qe/init_data.sh line 23:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data.sh line 27:
curl --location-trusted -T baseall.txt -u "${FE_USER}":"${FE_PASSWORD}" http://"${FE_HOST}":"${FE_WEB_PORT}"/api/"${FE_DB}"/baseall/_stream_load
                                                                                             ^------------^ SC2154 (warning): FE_WEB_PORT is referenced but not assigned.


In pytest/deploy/qe/init_data_broker.sh line 24:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data_broker.sh line 33:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_1 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE baseall COLUMNS TERMINATED BY '${separator}') ${broker_property}"
                                                                                                                           ^---------------^ SC2154 (warning): hdfs_data_path is referenced but not assigned.
                                                                                                                                                                                                                       ^----------------^ SC2154 (warning): broker_property is referenced but not assigned.


In pytest/deploy/qe/init_data_duplicate.sh line 23:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data_duplicate.sh line 32:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_1 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE baseall COLUMNS TERMINATED BY '${separator}') ${broker_property}"
                                                                                                                           ^---------------^ SC2154 (warning): hdfs_data_path is referenced but not assigned.
                                                                                                                                                                                                                       ^----------------^ SC2154 (warning): broker_property is referenced but not assigned.


In pytest/deploy/qe/init_data_list.sh line 23:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data_list.sh line 32:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_1 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE baseall COLUMNS TERMINATED BY '${separator}') ${broker_property}"
                                                                                                                           ^---------------^ SC2154 (warning): hdfs_data_path is referenced but not assigned.
                                                                                                                                                                                                                       ^----------------^ SC2154 (warning): broker_property is referenced but not assigned.


In pytest/deploy/qe/init_data_multi.sh line 23:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data_multi.sh line 32:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_1 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE baseall COLUMNS TERMINATED BY '${separator}') ${broker_property}"
                                                                                                                           ^---------------^ SC2154 (warning): hdfs_data_path is referenced but not assigned.
                                                                                                                                                                                                                       ^----------------^ SC2154 (warning): broker_property is referenced but not assigned.


In pytest/deploy/qe/init_data_stream.sh line 21:
cd ../../hdfs/data/qe/
^--------------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
cd ../../hdfs/data/qe/ || exit


In pytest/deploy/qe/init_data_stream.sh line 23:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data_stream.sh line 27:
curl --location-trusted -T baseall.txt -u "${FE_USER}":"${FE_PASSWORD}" http://"${FE_HOST}":"${FE_WEB_PORT}"/api/"${FE_DB}"/baseall/_stream_load
                                                                                             ^------------^ SC2154 (warning): FE_WEB_PORT is referenced but not assigned.


In pytest/deploy/qe/init_data_unique.sh line 23:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data_unique.sh line 32:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_1 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE baseall COLUMNS TERMINATED BY '${separator}') ${broker_property}"
                                                                                                                           ^---------------^ SC2154 (warning): hdfs_data_path is referenced but not assigned.
                                                                                                                                                                                                                       ^----------------^ SC2154 (warning): broker_property is referenced but not assigned.


In pytest/deploy/qe/init_ext.sh line 19:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e 'DROP DATABASE IF EXISTS test_query_qa'
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.


In pytest/deploy/qe/init_mysql.sh line 20:
mysql -h "${MYSQL_HOST}" -P"${MYSQL_PORT}" -u"${MYSQL_USER}" -p"${MYSQL_PASSWORD}" -e "DROP DATABASE IF EXISTS ${MYSQL_DB}"
          ^-----------^ SC2154 (warning): MYSQL_HOST is referenced but not assigned.
                            ^-----------^ SC2154 (warning): MYSQL_PORT is referenced but not assigned.
                                              ^-----------^ SC2154 (warning): MYSQL_USER is referenced but not assigned.
                                                                ^---------------^ SC2154 (warning): MYSQL_PASSWORD is referenced but not assigned.
                                                                                                               ^---------^ SC2154 (warning): MYSQL_DB is referenced but not assigned.


In pytest/deploy/qe/init_mysql.sh line 24:
mysql -h "${MYSQL_HOST}" -P"${MYSQL_PORT}" -u"${MYSQL_USER}" -p"${MYSQL_PASSWORD}" "${MYSQL_DB}" -e 'CREATE TABLE `baseall` ( `k1` tinyint(4) DEFAULT NULL, `k2` smallint(6) DEFAULT NULL, `k3` int(11) DEFAULT NULL, `k4` bigint(20) DEFAULT NULL, `k5` decimal(9,3) DEFAULT NULL, `k6` char(5) DEFAULT NULL, `k10` date DEFAULT NULL, `k11` datetime DEFAULT NULL, `k7` varchar(20) DEFAULT NULL, `k8` double DEFAULT NULL, `k9` float DEFAULT NULL)'
                                                                                                    ^-- SC2016 (info): Expressions don't expand in single quotes, use double quotes for that.


In pytest/deploy/qe/init_mysql.sh line 25:
mysql -h "${MYSQL_HOST}" -P"${MYSQL_PORT}" -u"${MYSQL_USER}" -p"${MYSQL_PASSWORD}" "${MYSQL_DB}" -e 'CREATE TABLE `bigtable` ( `k1` tinyint(4) DEFAULT NULL, `k2` smallint(6) DEFAULT NULL, `k3` int(11) DEFAULT NULL, `k4` bigint(20) DEFAULT NULL, `k5` decimal(9,3) DEFAULT NULL, `k6` char(5) DEFAULT NULL, `k10` date DEFAULT NULL, `k11` datetime DEFAULT NULL, `k7` varchar(20) DEFAULT NULL, `k8` double DEFAULT NULL, `k9` float DEFAULT NULL)'
                                                                                                    ^-- SC2016 (info): Expressions don't expand in single quotes, use double quotes for that.


In pytest/deploy/qe/init_mysql.sh line 26:
mysql -h "${MYSQL_HOST}" -P"${MYSQL_PORT}" -u"${MYSQL_USER}" -p"${MYSQL_PASSWORD}" "${MYSQL_DB}" -e 'CREATE TABLE `test` ( `k1` tinyint(4) DEFAULT NULL, `k2` smallint(6) DEFAULT NULL, `k3` int(11) DEFAULT NULL, `k4` bigint(20) DEFAULT NULL, `k5` decimal(9,3) DEFAULT NULL, `k6` char(5) DEFAULT NULL, `k10` date DEFAULT NULL, `k11` datetime DEFAULT NULL, `k7` varchar(20) DEFAULT NULL, `k8` double DEFAULT NULL, `k9` float DEFAULT NULL)'
                                                                                                    ^-- SC2016 (info): Expressions don't expand in single quotes, use double quotes for that.


In pytest/qe/palo2/data/init_lateral_view.sh line 19:
export hdfs_path=${hdfs_data_path}/qe/lateral_view
                 ^---------------^ SC2154 (warning): hdfs_data_path is referenced but not assigned.


In pytest/qe/palo2/data/init_lateral_view.sh line 21:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS test_lateral_view"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.


In pytest/qe/palo2/data/init_lateral_view.sh line 25:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL test_lateral_view.label_1 (DATA INFILE('${hdfs_path}/lateral_view_data') INTO TABLE lateral_view_data (k1,k2,k3,k4,k5,k6,k7,k8) set (k9=bitmap_from_string(k8))) ${broker_property}"
                                                                                                                                                                                                                                                             ^----------------^ SC2154 (warning): broker_property is referenced but not assigned.


In pytest/run_case.sh line 19:
export ROOT_PATH=$(pwd)
       ^-------^ SC2155 (warning): Declare and assign separately to avoid masking return values.


In pytest/run_case.sh line 26:
rm -f ${ROOT_PATH}/*.xml | tee
      ^----------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
rm -f "${ROOT_PATH}"/*.xml | tee


In pytest/run_case.sh line 27:
rm -f ${ROOT_PATH}/*.html | tee
      ^----------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
rm -f "${ROOT_PATH}"/*.html | tee


In pytest/run_case.sh line 28:
rm -rf ${REPORT_PATH}
       ^------------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
rm -rf "${REPORT_PATH}"


In pytest/run_case.sh line 29:
mkdir -p ${REPORT_PATH}
         ^------------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
mkdir -p "${REPORT_PATH}"


In pytest/run_case.sh line 31:
cd ${SYS_PATH}
^------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
   ^---------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
cd "${SYS_PATH}" || exit


In pytest/run_case.sh line 57:
    cd ${ENV_PATH}
    ^------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
       ^---------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
    cd "${ENV_PATH}" || exit


In pytest/run_case.sh line 67:
    ls ${case_file}
       ^----------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
    ls "${case_file}"


In pytest/run_case.sh line 94:
    cd ${SYS_PATH}
    ^------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
       ^---------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
    cd "${SYS_PATH}" || exit


In pytest/run_case.sh line 131:
    cd "${QE_PATH}"
    ^-------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${QE_PATH}" || exit


In pytest/run_case.sh line 136:
    cd "${QE_BUG_PATH}"
    ^-----------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${QE_BUG_PATH}" || exit


In pytest/run_case.sh line 193:
    cd "${SYS_PATH}"
    ^--------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}" || exit


In pytest/run_case.sh line 200:
    cd "${SYS_PATH}"/test_sys_routine_load/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_routine_load/ || exit


In pytest/run_case.sh line 207:
    cd "${SYS_PATH}"/test_sys_broker_load/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_broker_load/ || exit


In pytest/run_case.sh line 214:
    cd "${SYS_PATH}"/test_sys_backup_restore
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_backup_restore || exit


In pytest/run_case.sh line 224:
    cd "${SYS_PATH}"/test_sys_stream_load/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_stream_load/ || exit


In pytest/run_case.sh line 241:
    cd "${SYS_PATH}"/test_sys_alter_for_uniq/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_alter_for_uniq/ || exit


In pytest/run_case.sh line 258:
    cd "${SYS_PATH}"/test_sys_alter_for_duplicate/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_alter_for_duplicate/ || exit


In pytest/run_case.sh line 275:
    cd "${SYS_PATH}"/test_sys_alter_for_aggregate/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_alter_for_aggregate/ || exit


In pytest/run_case.sh line 283:
    cd "${SYS_PATH}"/test_sys_bitmap/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_bitmap/ || exit


In pytest/run_case.sh line 291:
    cd "${SYS_PATH}"/test_sys_export/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_export/ || exit


In pytest/run_case.sh line 296:
    cd "${SYS_PATH}"/test_sys_string
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_string || exit


In pytest/run_case.sh line 304:
    cd "${SYS_PATH}"/test_sys_array
    ^-----------------------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_array || exit


In pytest/run_restart_case.sh line 19:
export ROOT_PATH=$(pwd)
       ^-------^ SC2155 (warning): Declare and assign separately to avoid masking return values.


In pytest/run_restart_case.sh line 42:
cd "${SYS_PATH}"/test_scene
^-------------------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
cd "${SYS_PATH}"/test_scene || exit


In pytest/run_restart_case.sh line 48:
    cd "${SYS_PATH}"
    ^--------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}" || exit


In pytest/run_restart_case.sh line 55:
cd "${SYS_PATH}"
^--------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
cd "${SYS_PATH}" || exit

For more information:
  https://www.shellcheck.net/wiki/SC2154 -- FE_DB is referenced but not assig...
  https://www.shellcheck.net/wiki/SC2155 -- Declare and assign separately to ...
  https://www.shellcheck.net/wiki/SC2164 -- Use 'cd ... || exit' or 'cd ... |...
----------

You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
  # shellcheck disable=NNNN
above the line that contains the issue, where NNNN is the error code;
3. Add '-e NNNN' to the SHELLCHECK_OPTS setting in your .yml action file.

shfmt errors


'shfmt ' returned error 1 finding the following formatting issues:

----------
--- pytest/deploy/qe/init_data.sh.orig
+++ pytest/deploy/qe/init_data.sh
@@ -1,4 +1,4 @@
-#!/bin/sh	
+#!/bin/sh
 # Licensed to the Apache Software Foundation (ASF) under one
 # or more contributor license agreements.  See the NOTICE file
 # distributed with this work for additional information
@@ -32,4 +32,3 @@
 
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" "${FE_DB}" -e 'create table bigtable(k1 tinyint, k2 smallint, k3 int, k4 bigint, k5 decimal(9,3), k6 char(5), k10 date, k11 datetime, k7 varchar(20), k8 double max, k9 float sum) engine=olap distributed by hash(k1) buckets 5 properties("storage_type"="column")'
 curl --location-trusted -T baseall.txt -u "${FE_USER}":"${FE_PASSWORD}" http://"${FE_HOST}":"${FE_WEB_PORT}"/api/"${FE_DB}"/bigtable/_stream_load
-
--- pytest/deploy/qe/init_data_broker.sh.orig
+++ pytest/deploy/qe/init_data_broker.sh
@@ -20,7 +20,6 @@
 # hdfs_data_path='hdfs://xxxxxx'
 # broker_property="WITH BROKER 'hdfs' ('username'='root', 'password'='')"
 
-
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "create database ${FE_DB}"
 
@@ -29,7 +28,6 @@
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "create table ${FE_DB}.bigtable(k1 tinyint, k2 smallint, k3 int, k4 bigint, k5 decimal(9,3), k6 char(5), k10 date, k11 datetime, k7 varchar(20), k8 double max, k9 float sum) engine=olap distributed by hash(k1) buckets 5 properties('storage_type'='column')"
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "CREATE TABLE ${FE_DB}.all_type_table (tinyint_key tinyint(4) NULL, smallint_key smallint(6) NULL, int_key int(11) NOT NULL, bigint_key bigint(20) NULL, char_50_key char(50) NULL, character_key varchar(500) NOT NULL, char_key char(1) NULL, character_most_key varchar(65533) NULL, decimal_key decimal(20, 6) NULL, decimal_most_key decimal(27, 9) NULL, date_key date NULL, datetime_key datetime NULL, tinyint_value tinyint(4) SUM NULL, smallint_value smallint(6) SUM NULL, int_value int(11) SUM NULL, largeint_value largeint SUM NULL, char_50_value char(50) REPLACE NULL, character_value varchar(500) REPLACE NULL, char_value char(1) REPLACE NULL, character_most_value varchar(65533) REPLACE NULL, decimal_value decimal(20, 6) SUM NULL, decimal_most_value decimal(27, 9) SUM NULL, date_value_replace date REPLACE NULL, date_value_max date REPLACE NULL, date_value_min date REPLACE NULL, datetime_value_replace datetime REPLACE NULL, datetime_value_max datetime REPLACE NULL, datetime_value_min datetime REPLACE NULL, float_value float SUM NULL, double_value double SUM NULL) ENGINE=OLAP DISTRIBUTED BY HASH(tinyint_key) BUCKETS 5 PROPERTIES ('storage_type' = 'COLUMN')"
 
-
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_1 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE baseall COLUMNS TERMINATED BY '${separator}') ${broker_property}"
 
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_2 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE test COLUMNS TERMINATED BY '${separator}') ${broker_property}"
--- pytest/deploy/qe/init_data_list.sh.orig
+++ pytest/deploy/qe/init_data_list.sh
@@ -37,5 +37,4 @@
 
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_4 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE bigtable COLUMNS TERMINATED BY '${separator}') ${broker_property}"
 
-
 sleep 20
--- pytest/deploy/qe/init_data_stream.sh.orig
+++ pytest/deploy/qe/init_data_stream.sh
@@ -1,4 +1,4 @@
-#!/bin/sh	
+#!/bin/sh
 # Licensed to the Apache Software Foundation (ASF) under one
 # or more contributor license agreements.  See the NOTICE file
 # distributed with this work for additional information
@@ -32,4 +32,3 @@
 
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" "${FE_DB}" -e 'create table bigtable(k1 tinyint, k2 smallint, k3 int, k4 bigint, k5 decimal(9,3), k6 char(5), k10 date, k11 datetime, k7 varchar(20), k8 double max, k9 float sum) engine=olap distributed by hash(k1) buckets 5 properties("storage_type"="column")'
 curl --location-trusted -T baseall.txt -u "${FE_USER}":"${FE_PASSWORD}" http://"${FE_HOST}":"${FE_WEB_PORT}"/api/"${FE_DB}"/bigtable/_stream_load
-
--- pytest/download_data.sh.orig
+++ pytest/download_data.sh
@@ -16,11 +16,13 @@
 # specific language governing permissions and limitations
 # under the License.
 
-
 set -eo pipefail
 
 PYTEST_HOME=$(dirname "$0")
-PYTEST_HOME=$(cd "${PYTEST_HOME}"; pwd)
+PYTEST_HOME=$(
+    cd "${PYTEST_HOME}"
+    pwd
+)
 
 export PYTEST_HOME
 
@@ -39,7 +41,7 @@
     wget -O "${HDFS_DATA}" https://palo-qa.cdn.bcebos.com/pytest_data/hdfs_data.tar.gz
     rm -rf hdfs
     mkdir -p "${PYTEST_HOME}"/hdfs/
-    tar xzf "${HDFS_DATA}" -C "${PYTEST_HOME}"/hdfs/ 
+    tar xzf "${HDFS_DATA}" -C "${PYTEST_HOME}"/hdfs/
     echo "hdfs data download finished, need to put to hdfs"
 fi
 
@@ -69,6 +71,3 @@
     fi
     echo "local data download finished"
 fi
-
-
-
--- pytest/qe/palo2/data/init_lateral_view.sh.orig
+++ pytest/qe/palo2/data/init_lateral_view.sh
@@ -74,7 +74,7 @@
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e 'create table test_lateral_view.test_multi_partition_bucket(k1 int, k2 int, k3 int, k4 varchar(500), k5 varchar(500), k6 varchar(500), k7 varchar(500), k8 varchar(500), k9 bitmap bitmap_union) PARTITION BY RANGE(k1,k2) (partition p1 values less than ("-32768","-8"), partition p2 values less than ("0","0"),partition p3 values less than ("32768","8"),partition p4 values less than ("65536","18")) DISTRIBUTED BY HASH(k1,k2,k3,k4,k5,k6,k7,k8)'
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL test_lateral_view.label_18 (DATA INFILE('${hdfs_path}/lateral_view_data') INTO TABLE test_multi_partition_bucket (k1,k2,k3,k4,k5,k6,k7,k8) set (k9=bitmap_from_string(k8))) ${broker_property}"
 
-mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e  'create table test_lateral_view.test_explode_split_type(k1 int, k2 char(255), k3 varchar(500), k4 string) DISTRIBUTED BY HASH(k1)'
+mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e 'create table test_lateral_view.test_explode_split_type(k1 int, k2 char(255), k3 varchar(500), k4 string) DISTRIBUTED BY HASH(k1)'
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL test_lateral_view.label_19 (DATA INFILE('${hdfs_path}/lateral_view_data') INTO TABLE test_explode_split_type (k1,v2,v3,v4,v5,v6,v7,v8) set (k2=v4,k3=v4,k4=v4)) ${broker_property}"
 
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e 'create table test_lateral_view.test_explode_split_type_check(k1 int, k2 varchar(50)) DISTRIBUTED BY HASH(k1)'
@@ -85,4 +85,3 @@
 
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e 'create table test_lateral_view.lateral_view_multi_check(k1 int, k2 int, k3 int, k4 varchar(200), k5 bigint, k6 bigint, k7 double) DISTRIBUTED BY HASH(k1)'
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL test_lateral_view.label_22 (DATA INFILE('${hdfs_path}/lateral_view_multi_check') INTO TABLE lateral_view_multi_check (k1,k2,k3,k4,k5,k6,k7)) ${broker_property}"
-
--- pytest/run_case.sh.orig
+++ pytest/run_case.sh
@@ -46,13 +46,11 @@
 exec 3<>/tmp/fd1
 rm -rf /tmp/fd1
 
-for ((i=1;i<=PROCESS_NUM;i++))
-do
-        echo >&3
+for ((i = 1; i <= PROCESS_NUM; i++)); do
+    echo >&3
 done
 
-function check_cluster ()
-{
+function check_cluster() {
     # check if fe or be dead and start it
     cd ${ENV_PATH}
     python node_op.py -c '''
@@ -61,42 +59,37 @@
     '''
 }
 
-function pytest_execute ()
-{
+function pytest_execute() {
     case_file=$1
     ls ${case_file}
-    pytest -sv --junit-xml="${REPORT_PATH}"/"${case_file%.py}".xml --html="${REPORT_PATH}"/"${case_file%.py}".html  "${case_file}" --tb=native 2>&1 | tee "${case_file%.py}".log
+    pytest -sv --junit-xml="${REPORT_PATH}"/"${case_file%.py}".xml --html="${REPORT_PATH}"/"${case_file%.py}".html "${case_file}" --tb=native 2>&1 | tee "${case_file%.py}".log
     sleep 1
 }
 
-function case_execute ()
-{
-    for case_file in "$@"
-    do
+function case_execute() {
+    for case_file in "$@"; do
         read -r -u3
         {
             echo "run ${case_file}"
             pytest_execute "${case_file}"
             if [[ "${RESTART_BE_AUTO}" == true ]]; then
                 check_cluster
-            fi     
+            fi
             echo >&3
-        }&
+        } &
     done
 
 }
 
 # 执行时间较长，最先执行
-function long_time_case ()
-{
+function long_time_case() {
     case_files='test_sys_partition_complex_without_restart_be.py 
                 test_sys_storage_medium.py test_sys_delete_load.py'
     cd ${SYS_PATH}
     case_execute "${case_files}"
 }
 
-function query_case ()
-{
+function query_case() {
     # query case
     case_files='test_query_predicates.py 
                 test_query_others.py
@@ -137,8 +130,7 @@
     case_execute "${case_files}"
 }
 
-function sys_case ()
-{
+function sys_case() {
     # sys case
     case_files='test_sys_verify.py
                 test_sys_bloom_filter_a.py
@@ -290,7 +282,7 @@
                 test_select_into_query.py'
     cd "${SYS_PATH}"/test_sys_export/
     case_execute "${case_files}"
-   
+
     # sys/test_sys_string
     case_files='test_sys_string_basic.py'
     cd "${SYS_PATH}"/test_sys_string
--- pytest/run_restart_case.sh.orig
+++ pytest/run_restart_case.sh
@@ -29,11 +29,10 @@
     testsuite=normal
 fi
 
-function pytest_execute()
-{
+function pytest_execute() {
     case_file=$1
     ls "${case_file}"
-    pytest -sv --junit-xml="${REPORT_PATH}"/"${case_file%.py}".xml --html="${REPORT_PATH}"/"${case_file%.py}".html  "${case_file}" --tb=native 2>&1 | tee "${case_file%.py}".log
+    pytest -sv --junit-xml="${REPORT_PATH}"/"${case_file%.py}".xml --html="${REPORT_PATH}"/"${case_file%.py}".html "${case_file}" --tb=native 2>&1 | tee "${case_file%.py}".log
     sleep 1
 }
 
@@ -43,8 +42,7 @@
 pytest_execute "${case}"
 
 cases='test_sys_update_restart.py test_sys_partition_complex_with_restart_be.py test_sys_restart.py test_sys_binlog_restart.py test_sys_resource_tag.py'
-for i in ${cases}
-do
+for i in ${cases}; do
     cd "${SYS_PATH}"
     echo "${i}"
     pytest_execute "${i}"
----------

You can reformat the above files to meet shfmt's requirements by typing:

  shfmt  -w filename

github-actions

clang-tidy made some suggestions

github-actions · 2023-11-22T10:33:42Z

be/test/olap/compaction_delete_bitmap_calculator_test.cpp

+// specific language governing permissions and limitations
+// under the License.
+
+#include <gen_cpp/AgentService_types.h>


warning: 'gen_cpp/AgentService_types.h' file not found [clang-diagnostic-error]

#include <gen_cpp/AgentService_types.h> ^

github-actions · 2023-11-22T10:33:42Z

be/test/testutil/http_utils.cpp

+// specific language governing permissions and limitations
+// under the License.
+
+#include "testutil/http_utils.h"


warning: 'testutil/http_utils.h' file not found [clang-diagnostic-error]

#include "testutil/http_utils.h" ^

github-actions · 2023-11-22T10:33:42Z

be/test/testutil/http_utils.h

+
+#pragma once
+
+#include "http/http_client.h"


warning: 'http/http_client.h' file not found [clang-diagnostic-error]

#include "http/http_client.h" ^

be/test/util/debug_points_test.cpp

eldenmoon · 2023-11-22T12:16:06Z

run buildall

eldenmoon · 2023-11-22T13:24:14Z

run buildall

eldenmoon · 2023-11-22T13:25:09Z

run buildall

github-actions · 2023-11-22T13:25:56Z

`sh-checker report`

To get the full details, please check in the job output.

shellcheck errors


'shellcheck ' returned error 1 finding the following syntactical issues:

----------

In pytest/deploy/qe/init_data.sh line 23:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data.sh line 27:
curl --location-trusted -T baseall.txt -u "${FE_USER}":"${FE_PASSWORD}" http://"${FE_HOST}":"${FE_WEB_PORT}"/api/"${FE_DB}"/baseall/_stream_load
                                                                                             ^------------^ SC2154 (warning): FE_WEB_PORT is referenced but not assigned.


In pytest/deploy/qe/init_data_broker.sh line 24:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data_broker.sh line 33:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_1 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE baseall COLUMNS TERMINATED BY '${separator}') ${broker_property}"
                                                                                                                           ^---------------^ SC2154 (warning): hdfs_data_path is referenced but not assigned.
                                                                                                                                                                                                                       ^----------------^ SC2154 (warning): broker_property is referenced but not assigned.


In pytest/deploy/qe/init_data_duplicate.sh line 23:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data_duplicate.sh line 32:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_1 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE baseall COLUMNS TERMINATED BY '${separator}') ${broker_property}"
                                                                                                                           ^---------------^ SC2154 (warning): hdfs_data_path is referenced but not assigned.
                                                                                                                                                                                                                       ^----------------^ SC2154 (warning): broker_property is referenced but not assigned.


In pytest/deploy/qe/init_data_list.sh line 23:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data_list.sh line 32:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_1 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE baseall COLUMNS TERMINATED BY '${separator}') ${broker_property}"
                                                                                                                           ^---------------^ SC2154 (warning): hdfs_data_path is referenced but not assigned.
                                                                                                                                                                                                                       ^----------------^ SC2154 (warning): broker_property is referenced but not assigned.


In pytest/deploy/qe/init_data_multi.sh line 23:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data_multi.sh line 32:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_1 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE baseall COLUMNS TERMINATED BY '${separator}') ${broker_property}"
                                                                                                                           ^---------------^ SC2154 (warning): hdfs_data_path is referenced but not assigned.
                                                                                                                                                                                                                       ^----------------^ SC2154 (warning): broker_property is referenced but not assigned.


In pytest/deploy/qe/init_data_stream.sh line 21:
cd ../../hdfs/data/qe/
^--------------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
cd ../../hdfs/data/qe/ || exit


In pytest/deploy/qe/init_data_stream.sh line 23:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data_stream.sh line 27:
curl --location-trusted -T baseall.txt -u "${FE_USER}":"${FE_PASSWORD}" http://"${FE_HOST}":"${FE_WEB_PORT}"/api/"${FE_DB}"/baseall/_stream_load
                                                                                             ^------------^ SC2154 (warning): FE_WEB_PORT is referenced but not assigned.


In pytest/deploy/qe/init_data_unique.sh line 23:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data_unique.sh line 32:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_1 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE baseall COLUMNS TERMINATED BY '${separator}') ${broker_property}"
                                                                                                                           ^---------------^ SC2154 (warning): hdfs_data_path is referenced but not assigned.
                                                                                                                                                                                                                       ^----------------^ SC2154 (warning): broker_property is referenced but not assigned.


In pytest/deploy/qe/init_ext.sh line 19:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e 'DROP DATABASE IF EXISTS test_query_qa'
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.


In pytest/deploy/qe/init_mysql.sh line 20:
mysql -h "${MYSQL_HOST}" -P"${MYSQL_PORT}" -u"${MYSQL_USER}" -p"${MYSQL_PASSWORD}" -e "DROP DATABASE IF EXISTS ${MYSQL_DB}"
          ^-----------^ SC2154 (warning): MYSQL_HOST is referenced but not assigned.
                            ^-----------^ SC2154 (warning): MYSQL_PORT is referenced but not assigned.
                                              ^-----------^ SC2154 (warning): MYSQL_USER is referenced but not assigned.
                                                                ^---------------^ SC2154 (warning): MYSQL_PASSWORD is referenced but not assigned.
                                                                                                               ^---------^ SC2154 (warning): MYSQL_DB is referenced but not assigned.


In pytest/deploy/qe/init_mysql.sh line 24:
mysql -h "${MYSQL_HOST}" -P"${MYSQL_PORT}" -u"${MYSQL_USER}" -p"${MYSQL_PASSWORD}" "${MYSQL_DB}" -e 'CREATE TABLE `baseall` ( `k1` tinyint(4) DEFAULT NULL, `k2` smallint(6) DEFAULT NULL, `k3` int(11) DEFAULT NULL, `k4` bigint(20) DEFAULT NULL, `k5` decimal(9,3) DEFAULT NULL, `k6` char(5) DEFAULT NULL, `k10` date DEFAULT NULL, `k11` datetime DEFAULT NULL, `k7` varchar(20) DEFAULT NULL, `k8` double DEFAULT NULL, `k9` float DEFAULT NULL)'
                                                                                                    ^-- SC2016 (info): Expressions don't expand in single quotes, use double quotes for that.


In pytest/deploy/qe/init_mysql.sh line 25:
mysql -h "${MYSQL_HOST}" -P"${MYSQL_PORT}" -u"${MYSQL_USER}" -p"${MYSQL_PASSWORD}" "${MYSQL_DB}" -e 'CREATE TABLE `bigtable` ( `k1` tinyint(4) DEFAULT NULL, `k2` smallint(6) DEFAULT NULL, `k3` int(11) DEFAULT NULL, `k4` bigint(20) DEFAULT NULL, `k5` decimal(9,3) DEFAULT NULL, `k6` char(5) DEFAULT NULL, `k10` date DEFAULT NULL, `k11` datetime DEFAULT NULL, `k7` varchar(20) DEFAULT NULL, `k8` double DEFAULT NULL, `k9` float DEFAULT NULL)'
                                                                                                    ^-- SC2016 (info): Expressions don't expand in single quotes, use double quotes for that.


In pytest/deploy/qe/init_mysql.sh line 26:
mysql -h "${MYSQL_HOST}" -P"${MYSQL_PORT}" -u"${MYSQL_USER}" -p"${MYSQL_PASSWORD}" "${MYSQL_DB}" -e 'CREATE TABLE `test` ( `k1` tinyint(4) DEFAULT NULL, `k2` smallint(6) DEFAULT NULL, `k3` int(11) DEFAULT NULL, `k4` bigint(20) DEFAULT NULL, `k5` decimal(9,3) DEFAULT NULL, `k6` char(5) DEFAULT NULL, `k10` date DEFAULT NULL, `k11` datetime DEFAULT NULL, `k7` varchar(20) DEFAULT NULL, `k8` double DEFAULT NULL, `k9` float DEFAULT NULL)'
                                                                                                    ^-- SC2016 (info): Expressions don't expand in single quotes, use double quotes for that.


In pytest/qe/palo2/data/init_lateral_view.sh line 19:
export hdfs_path=${hdfs_data_path}/qe/lateral_view
                 ^---------------^ SC2154 (warning): hdfs_data_path is referenced but not assigned.


In pytest/qe/palo2/data/init_lateral_view.sh line 21:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS test_lateral_view"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.


In pytest/qe/palo2/data/init_lateral_view.sh line 25:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL test_lateral_view.label_1 (DATA INFILE('${hdfs_path}/lateral_view_data') INTO TABLE lateral_view_data (k1,k2,k3,k4,k5,k6,k7,k8) set (k9=bitmap_from_string(k8))) ${broker_property}"
                                                                                                                                                                                                                                                             ^----------------^ SC2154 (warning): broker_property is referenced but not assigned.


In pytest/run_case.sh line 19:
export ROOT_PATH=$(pwd)
       ^-------^ SC2155 (warning): Declare and assign separately to avoid masking return values.


In pytest/run_case.sh line 26:
rm -f ${ROOT_PATH}/*.xml | tee
      ^----------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
rm -f "${ROOT_PATH}"/*.xml | tee


In pytest/run_case.sh line 27:
rm -f ${ROOT_PATH}/*.html | tee
      ^----------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
rm -f "${ROOT_PATH}"/*.html | tee


In pytest/run_case.sh line 28:
rm -rf ${REPORT_PATH}
       ^------------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
rm -rf "${REPORT_PATH}"


In pytest/run_case.sh line 29:
mkdir -p ${REPORT_PATH}
         ^------------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
mkdir -p "${REPORT_PATH}"


In pytest/run_case.sh line 31:
cd ${SYS_PATH}
^------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
   ^---------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
cd "${SYS_PATH}" || exit


In pytest/run_case.sh line 57:
    cd ${ENV_PATH}
    ^------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
       ^---------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
    cd "${ENV_PATH}" || exit


In pytest/run_case.sh line 67:
    ls ${case_file}
       ^----------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
    ls "${case_file}"


In pytest/run_case.sh line 94:
    cd ${SYS_PATH}
    ^------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
       ^---------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
    cd "${SYS_PATH}" || exit


In pytest/run_case.sh line 131:
    cd "${QE_PATH}"
    ^-------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${QE_PATH}" || exit


In pytest/run_case.sh line 136:
    cd "${QE_BUG_PATH}"
    ^-----------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${QE_BUG_PATH}" || exit


In pytest/run_case.sh line 193:
    cd "${SYS_PATH}"
    ^--------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}" || exit


In pytest/run_case.sh line 200:
    cd "${SYS_PATH}"/test_sys_routine_load/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_routine_load/ || exit


In pytest/run_case.sh line 207:
    cd "${SYS_PATH}"/test_sys_broker_load/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_broker_load/ || exit


In pytest/run_case.sh line 214:
    cd "${SYS_PATH}"/test_sys_backup_restore
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_backup_restore || exit


In pytest/run_case.sh line 224:
    cd "${SYS_PATH}"/test_sys_stream_load/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_stream_load/ || exit


In pytest/run_case.sh line 241:
    cd "${SYS_PATH}"/test_sys_alter_for_uniq/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_alter_for_uniq/ || exit


In pytest/run_case.sh line 258:
    cd "${SYS_PATH}"/test_sys_alter_for_duplicate/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_alter_for_duplicate/ || exit


In pytest/run_case.sh line 275:
    cd "${SYS_PATH}"/test_sys_alter_for_aggregate/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_alter_for_aggregate/ || exit


In pytest/run_case.sh line 283:
    cd "${SYS_PATH}"/test_sys_bitmap/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_bitmap/ || exit


In pytest/run_case.sh line 291:
    cd "${SYS_PATH}"/test_sys_export/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_export/ || exit


In pytest/run_case.sh line 296:
    cd "${SYS_PATH}"/test_sys_string
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_string || exit


In pytest/run_case.sh line 304:
    cd "${SYS_PATH}"/test_sys_array
    ^-----------------------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_array || exit


In pytest/run_restart_case.sh line 19:
export ROOT_PATH=$(pwd)
       ^-------^ SC2155 (warning): Declare and assign separately to avoid masking return values.


In pytest/run_restart_case.sh line 42:
cd "${SYS_PATH}"/test_scene
^-------------------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
cd "${SYS_PATH}"/test_scene || exit


In pytest/run_restart_case.sh line 48:
    cd "${SYS_PATH}"
    ^--------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}" || exit


In pytest/run_restart_case.sh line 55:
cd "${SYS_PATH}"
^--------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
cd "${SYS_PATH}" || exit

For more information:
  https://www.shellcheck.net/wiki/SC2154 -- FE_DB is referenced but not assig...
  https://www.shellcheck.net/wiki/SC2155 -- Declare and assign separately to ...
  https://www.shellcheck.net/wiki/SC2164 -- Use 'cd ... || exit' or 'cd ... |...
----------

You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
  # shellcheck disable=NNNN
above the line that contains the issue, where NNNN is the error code;
3. Add '-e NNNN' to the SHELLCHECK_OPTS setting in your .yml action file.

shfmt errors


'shfmt ' returned error 1 finding the following formatting issues:

----------
--- pytest/deploy/qe/init_data.sh.orig
+++ pytest/deploy/qe/init_data.sh
@@ -1,4 +1,4 @@
-#!/bin/sh	
+#!/bin/sh
 # Licensed to the Apache Software Foundation (ASF) under one
 # or more contributor license agreements.  See the NOTICE file
 # distributed with this work for additional information
@@ -32,4 +32,3 @@
 
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" "${FE_DB}" -e 'create table bigtable(k1 tinyint, k2 smallint, k3 int, k4 bigint, k5 decimal(9,3), k6 char(5), k10 date, k11 datetime, k7 varchar(20), k8 double max, k9 float sum) engine=olap distributed by hash(k1) buckets 5 properties("storage_type"="column")'
 curl --location-trusted -T baseall.txt -u "${FE_USER}":"${FE_PASSWORD}" http://"${FE_HOST}":"${FE_WEB_PORT}"/api/"${FE_DB}"/bigtable/_stream_load
-
--- pytest/deploy/qe/init_data_broker.sh.orig
+++ pytest/deploy/qe/init_data_broker.sh
@@ -20,7 +20,6 @@
 # hdfs_data_path='hdfs://xxxxxx'
 # broker_property="WITH BROKER 'hdfs' ('username'='root', 'password'='')"
 
-
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "create database ${FE_DB}"
 
@@ -29,7 +28,6 @@
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "create table ${FE_DB}.bigtable(k1 tinyint, k2 smallint, k3 int, k4 bigint, k5 decimal(9,3), k6 char(5), k10 date, k11 datetime, k7 varchar(20), k8 double max, k9 float sum) engine=olap distributed by hash(k1) buckets 5 properties('storage_type'='column')"
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "CREATE TABLE ${FE_DB}.all_type_table (tinyint_key tinyint(4) NULL, smallint_key smallint(6) NULL, int_key int(11) NOT NULL, bigint_key bigint(20) NULL, char_50_key char(50) NULL, character_key varchar(500) NOT NULL, char_key char(1) NULL, character_most_key varchar(65533) NULL, decimal_key decimal(20, 6) NULL, decimal_most_key decimal(27, 9) NULL, date_key date NULL, datetime_key datetime NULL, tinyint_value tinyint(4) SUM NULL, smallint_value smallint(6) SUM NULL, int_value int(11) SUM NULL, largeint_value largeint SUM NULL, char_50_value char(50) REPLACE NULL, character_value varchar(500) REPLACE NULL, char_value char(1) REPLACE NULL, character_most_value varchar(65533) REPLACE NULL, decimal_value decimal(20, 6) SUM NULL, decimal_most_value decimal(27, 9) SUM NULL, date_value_replace date REPLACE NULL, date_value_max date REPLACE NULL, date_value_min date REPLACE NULL, datetime_value_replace datetime REPLACE NULL, datetime_value_max datetime REPLACE NULL, datetime_value_min datetime REPLACE NULL, float_value float SUM NULL, double_value double SUM NULL) ENGINE=OLAP DISTRIBUTED BY HASH(tinyint_key) BUCKETS 5 PROPERTIES ('storage_type' = 'COLUMN')"
 
-
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_1 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE baseall COLUMNS TERMINATED BY '${separator}') ${broker_property}"
 
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_2 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE test COLUMNS TERMINATED BY '${separator}') ${broker_property}"
--- pytest/deploy/qe/init_data_list.sh.orig
+++ pytest/deploy/qe/init_data_list.sh
@@ -37,5 +37,4 @@
 
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_4 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE bigtable COLUMNS TERMINATED BY '${separator}') ${broker_property}"
 
-
 sleep 20
--- pytest/deploy/qe/init_data_stream.sh.orig
+++ pytest/deploy/qe/init_data_stream.sh
@@ -1,4 +1,4 @@
-#!/bin/sh	
+#!/bin/sh
 # Licensed to the Apache Software Foundation (ASF) under one
 # or more contributor license agreements.  See the NOTICE file
 # distributed with this work for additional information
@@ -32,4 +32,3 @@
 
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" "${FE_DB}" -e 'create table bigtable(k1 tinyint, k2 smallint, k3 int, k4 bigint, k5 decimal(9,3), k6 char(5), k10 date, k11 datetime, k7 varchar(20), k8 double max, k9 float sum) engine=olap distributed by hash(k1) buckets 5 properties("storage_type"="column")'
 curl --location-trusted -T baseall.txt -u "${FE_USER}":"${FE_PASSWORD}" http://"${FE_HOST}":"${FE_WEB_PORT}"/api/"${FE_DB}"/bigtable/_stream_load
-
--- pytest/download_data.sh.orig
+++ pytest/download_data.sh
@@ -16,11 +16,13 @@
 # specific language governing permissions and limitations
 # under the License.
 
-
 set -eo pipefail
 
 PYTEST_HOME=$(dirname "$0")
-PYTEST_HOME=$(cd "${PYTEST_HOME}"; pwd)
+PYTEST_HOME=$(
+    cd "${PYTEST_HOME}"
+    pwd
+)
 
 export PYTEST_HOME
 
@@ -39,7 +41,7 @@
     wget -O "${HDFS_DATA}" https://palo-qa.cdn.bcebos.com/pytest_data/hdfs_data.tar.gz
     rm -rf hdfs
     mkdir -p "${PYTEST_HOME}"/hdfs/
-    tar xzf "${HDFS_DATA}" -C "${PYTEST_HOME}"/hdfs/ 
+    tar xzf "${HDFS_DATA}" -C "${PYTEST_HOME}"/hdfs/
     echo "hdfs data download finished, need to put to hdfs"
 fi
 
@@ -69,6 +71,3 @@
     fi
     echo "local data download finished"
 fi
-
-
-
--- pytest/qe/palo2/data/init_lateral_view.sh.orig
+++ pytest/qe/palo2/data/init_lateral_view.sh
@@ -74,7 +74,7 @@
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e 'create table test_lateral_view.test_multi_partition_bucket(k1 int, k2 int, k3 int, k4 varchar(500), k5 varchar(500), k6 varchar(500), k7 varchar(500), k8 varchar(500), k9 bitmap bitmap_union) PARTITION BY RANGE(k1,k2) (partition p1 values less than ("-32768","-8"), partition p2 values less than ("0","0"),partition p3 values less than ("32768","8"),partition p4 values less than ("65536","18")) DISTRIBUTED BY HASH(k1,k2,k3,k4,k5,k6,k7,k8)'
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL test_lateral_view.label_18 (DATA INFILE('${hdfs_path}/lateral_view_data') INTO TABLE test_multi_partition_bucket (k1,k2,k3,k4,k5,k6,k7,k8) set (k9=bitmap_from_string(k8))) ${broker_property}"
 
-mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e  'create table test_lateral_view.test_explode_split_type(k1 int, k2 char(255), k3 varchar(500), k4 string) DISTRIBUTED BY HASH(k1)'
+mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e 'create table test_lateral_view.test_explode_split_type(k1 int, k2 char(255), k3 varchar(500), k4 string) DISTRIBUTED BY HASH(k1)'
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL test_lateral_view.label_19 (DATA INFILE('${hdfs_path}/lateral_view_data') INTO TABLE test_explode_split_type (k1,v2,v3,v4,v5,v6,v7,v8) set (k2=v4,k3=v4,k4=v4)) ${broker_property}"
 
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e 'create table test_lateral_view.test_explode_split_type_check(k1 int, k2 varchar(50)) DISTRIBUTED BY HASH(k1)'
@@ -85,4 +85,3 @@
 
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e 'create table test_lateral_view.lateral_view_multi_check(k1 int, k2 int, k3 int, k4 varchar(200), k5 bigint, k6 bigint, k7 double) DISTRIBUTED BY HASH(k1)'
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL test_lateral_view.label_22 (DATA INFILE('${hdfs_path}/lateral_view_multi_check') INTO TABLE lateral_view_multi_check (k1,k2,k3,k4,k5,k6,k7)) ${broker_property}"
-
--- pytest/run_case.sh.orig
+++ pytest/run_case.sh
@@ -46,13 +46,11 @@
 exec 3<>/tmp/fd1
 rm -rf /tmp/fd1
 
-for ((i=1;i<=PROCESS_NUM;i++))
-do
-        echo >&3
+for ((i = 1; i <= PROCESS_NUM; i++)); do
+    echo >&3
 done
 
-function check_cluster ()
-{
+function check_cluster() {
     # check if fe or be dead and start it
     cd ${ENV_PATH}
     python node_op.py -c '''
@@ -61,42 +59,37 @@
     '''
 }
 
-function pytest_execute ()
-{
+function pytest_execute() {
     case_file=$1
     ls ${case_file}
-    pytest -sv --junit-xml="${REPORT_PATH}"/"${case_file%.py}".xml --html="${REPORT_PATH}"/"${case_file%.py}".html  "${case_file}" --tb=native 2>&1 | tee "${case_file%.py}".log
+    pytest -sv --junit-xml="${REPORT_PATH}"/"${case_file%.py}".xml --html="${REPORT_PATH}"/"${case_file%.py}".html "${case_file}" --tb=native 2>&1 | tee "${case_file%.py}".log
     sleep 1
 }
 
-function case_execute ()
-{
-    for case_file in "$@"
-    do
+function case_execute() {
+    for case_file in "$@"; do
         read -r -u3
         {
             echo "run ${case_file}"
             pytest_execute "${case_file}"
             if [[ "${RESTART_BE_AUTO}" == true ]]; then
                 check_cluster
-            fi     
+            fi
             echo >&3
-        }&
+        } &
     done
 
 }
 
 # 执行时间较长，最先执行
-function long_time_case ()
-{
+function long_time_case() {
     case_files='test_sys_partition_complex_without_restart_be.py 
                 test_sys_storage_medium.py test_sys_delete_load.py'
     cd ${SYS_PATH}
     case_execute "${case_files}"
 }
 
-function query_case ()
-{
+function query_case() {
     # query case
     case_files='test_query_predicates.py 
                 test_query_others.py
@@ -137,8 +130,7 @@
     case_execute "${case_files}"
 }
 
-function sys_case ()
-{
+function sys_case() {
     # sys case
     case_files='test_sys_verify.py
                 test_sys_bloom_filter_a.py
@@ -290,7 +282,7 @@
                 test_select_into_query.py'
     cd "${SYS_PATH}"/test_sys_export/
     case_execute "${case_files}"
-   
+
     # sys/test_sys_string
     case_files='test_sys_string_basic.py'
     cd "${SYS_PATH}"/test_sys_string
--- pytest/run_restart_case.sh.orig
+++ pytest/run_restart_case.sh
@@ -29,11 +29,10 @@
     testsuite=normal
 fi
 
-function pytest_execute()
-{
+function pytest_execute() {
     case_file=$1
     ls "${case_file}"
-    pytest -sv --junit-xml="${REPORT_PATH}"/"${case_file%.py}".xml --html="${REPORT_PATH}"/"${case_file%.py}".html  "${case_file}" --tb=native 2>&1 | tee "${case_file%.py}".log
+    pytest -sv --junit-xml="${REPORT_PATH}"/"${case_file%.py}".xml --html="${REPORT_PATH}"/"${case_file%.py}".html "${case_file}" --tb=native 2>&1 | tee "${case_file%.py}".log
     sleep 1
 }
 
@@ -43,8 +42,7 @@
 pytest_execute "${case}"
 
 cases='test_sys_update_restart.py test_sys_partition_complex_with_restart_be.py test_sys_restart.py test_sys_binlog_restart.py test_sys_resource_tag.py'
-for i in ${cases}
-do
+for i in ${cases}; do
     cd "${SYS_PATH}"
     echo "${i}"
     pytest_execute "${i}"
----------

You can reformat the above files to meet shfmt's requirements by typing:

  shfmt  -w filename

eldenmoon · 2023-11-23T02:31:32Z

run buildall

hello-stephen · 2023-11-23T10:14:02Z

run buildall

eldenmoon · 2023-11-23T10:42:15Z

run buildall

github-actions · 2023-11-23T10:42:16Z

`sh-checker report`

To get the full details, please check in the job output.

shellcheck errors


'shellcheck ' returned error 1 finding the following syntactical issues:

----------

In pytest/deploy/qe/init_data.sh line 23:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data.sh line 27:
curl --location-trusted -T baseall.txt -u "${FE_USER}":"${FE_PASSWORD}" http://"${FE_HOST}":"${FE_WEB_PORT}"/api/"${FE_DB}"/baseall/_stream_load
                                                                                             ^------------^ SC2154 (warning): FE_WEB_PORT is referenced but not assigned.


In pytest/deploy/qe/init_data_broker.sh line 24:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data_broker.sh line 33:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_1 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE baseall COLUMNS TERMINATED BY '${separator}') ${broker_property}"
                                                                                                                           ^---------------^ SC2154 (warning): hdfs_data_path is referenced but not assigned.
                                                                                                                                                                                                                       ^----------------^ SC2154 (warning): broker_property is referenced but not assigned.


In pytest/deploy/qe/init_data_duplicate.sh line 23:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data_duplicate.sh line 32:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_1 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE baseall COLUMNS TERMINATED BY '${separator}') ${broker_property}"
                                                                                                                           ^---------------^ SC2154 (warning): hdfs_data_path is referenced but not assigned.
                                                                                                                                                                                                                       ^----------------^ SC2154 (warning): broker_property is referenced but not assigned.


In pytest/deploy/qe/init_data_list.sh line 23:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data_list.sh line 32:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_1 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE baseall COLUMNS TERMINATED BY '${separator}') ${broker_property}"
                                                                                                                           ^---------------^ SC2154 (warning): hdfs_data_path is referenced but not assigned.
                                                                                                                                                                                                                       ^----------------^ SC2154 (warning): broker_property is referenced but not assigned.


In pytest/deploy/qe/init_data_multi.sh line 23:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data_multi.sh line 32:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_1 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE baseall COLUMNS TERMINATED BY '${separator}') ${broker_property}"
                                                                                                                           ^---------------^ SC2154 (warning): hdfs_data_path is referenced but not assigned.
                                                                                                                                                                                                                       ^----------------^ SC2154 (warning): broker_property is referenced but not assigned.


In pytest/deploy/qe/init_data_stream.sh line 21:
cd ../../hdfs/data/qe/
^--------------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
cd ../../hdfs/data/qe/ || exit


In pytest/deploy/qe/init_data_stream.sh line 23:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data_stream.sh line 27:
curl --location-trusted -T baseall.txt -u "${FE_USER}":"${FE_PASSWORD}" http://"${FE_HOST}":"${FE_WEB_PORT}"/api/"${FE_DB}"/baseall/_stream_load
                                                                                             ^------------^ SC2154 (warning): FE_WEB_PORT is referenced but not assigned.


In pytest/deploy/qe/init_data_unique.sh line 23:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.
                                                                                                         ^------^ SC2154 (warning): FE_DB is referenced but not assigned.


In pytest/deploy/qe/init_data_unique.sh line 32:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_1 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE baseall COLUMNS TERMINATED BY '${separator}') ${broker_property}"
                                                                                                                           ^---------------^ SC2154 (warning): hdfs_data_path is referenced but not assigned.
                                                                                                                                                                                                                       ^----------------^ SC2154 (warning): broker_property is referenced but not assigned.


In pytest/deploy/qe/init_ext.sh line 19:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e 'DROP DATABASE IF EXISTS test_query_qa'
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.


In pytest/deploy/qe/init_mysql.sh line 20:
mysql -h "${MYSQL_HOST}" -P"${MYSQL_PORT}" -u"${MYSQL_USER}" -p"${MYSQL_PASSWORD}" -e "DROP DATABASE IF EXISTS ${MYSQL_DB}"
          ^-----------^ SC2154 (warning): MYSQL_HOST is referenced but not assigned.
                            ^-----------^ SC2154 (warning): MYSQL_PORT is referenced but not assigned.
                                              ^-----------^ SC2154 (warning): MYSQL_USER is referenced but not assigned.
                                                                ^---------------^ SC2154 (warning): MYSQL_PASSWORD is referenced but not assigned.
                                                                                                               ^---------^ SC2154 (warning): MYSQL_DB is referenced but not assigned.


In pytest/deploy/qe/init_mysql.sh line 24:
mysql -h "${MYSQL_HOST}" -P"${MYSQL_PORT}" -u"${MYSQL_USER}" -p"${MYSQL_PASSWORD}" "${MYSQL_DB}" -e 'CREATE TABLE `baseall` ( `k1` tinyint(4) DEFAULT NULL, `k2` smallint(6) DEFAULT NULL, `k3` int(11) DEFAULT NULL, `k4` bigint(20) DEFAULT NULL, `k5` decimal(9,3) DEFAULT NULL, `k6` char(5) DEFAULT NULL, `k10` date DEFAULT NULL, `k11` datetime DEFAULT NULL, `k7` varchar(20) DEFAULT NULL, `k8` double DEFAULT NULL, `k9` float DEFAULT NULL)'
                                                                                                    ^-- SC2016 (info): Expressions don't expand in single quotes, use double quotes for that.


In pytest/deploy/qe/init_mysql.sh line 25:
mysql -h "${MYSQL_HOST}" -P"${MYSQL_PORT}" -u"${MYSQL_USER}" -p"${MYSQL_PASSWORD}" "${MYSQL_DB}" -e 'CREATE TABLE `bigtable` ( `k1` tinyint(4) DEFAULT NULL, `k2` smallint(6) DEFAULT NULL, `k3` int(11) DEFAULT NULL, `k4` bigint(20) DEFAULT NULL, `k5` decimal(9,3) DEFAULT NULL, `k6` char(5) DEFAULT NULL, `k10` date DEFAULT NULL, `k11` datetime DEFAULT NULL, `k7` varchar(20) DEFAULT NULL, `k8` double DEFAULT NULL, `k9` float DEFAULT NULL)'
                                                                                                    ^-- SC2016 (info): Expressions don't expand in single quotes, use double quotes for that.


In pytest/deploy/qe/init_mysql.sh line 26:
mysql -h "${MYSQL_HOST}" -P"${MYSQL_PORT}" -u"${MYSQL_USER}" -p"${MYSQL_PASSWORD}" "${MYSQL_DB}" -e 'CREATE TABLE `test` ( `k1` tinyint(4) DEFAULT NULL, `k2` smallint(6) DEFAULT NULL, `k3` int(11) DEFAULT NULL, `k4` bigint(20) DEFAULT NULL, `k5` decimal(9,3) DEFAULT NULL, `k6` char(5) DEFAULT NULL, `k10` date DEFAULT NULL, `k11` datetime DEFAULT NULL, `k7` varchar(20) DEFAULT NULL, `k8` double DEFAULT NULL, `k9` float DEFAULT NULL)'
                                                                                                    ^-- SC2016 (info): Expressions don't expand in single quotes, use double quotes for that.


In pytest/qe/palo2/data/init_lateral_view.sh line 19:
export hdfs_path=${hdfs_data_path}/qe/lateral_view
                 ^---------------^ SC2154 (warning): hdfs_data_path is referenced but not assigned.


In pytest/qe/palo2/data/init_lateral_view.sh line 21:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS test_lateral_view"
          ^--------^ SC2154 (warning): FE_HOST is referenced but not assigned.
                         ^--------------^ SC2154 (warning): FE_QUERY_PORT is referenced but not assigned.
                                              ^--------^ SC2154 (warning): FE_USER is referenced but not assigned.
                                                             ^------------^ SC2154 (warning): FE_PASSWORD is referenced but not assigned.


In pytest/qe/palo2/data/init_lateral_view.sh line 25:
mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL test_lateral_view.label_1 (DATA INFILE('${hdfs_path}/lateral_view_data') INTO TABLE lateral_view_data (k1,k2,k3,k4,k5,k6,k7,k8) set (k9=bitmap_from_string(k8))) ${broker_property}"
                                                                                                                                                                                                                                                             ^----------------^ SC2154 (warning): broker_property is referenced but not assigned.


In pytest/run_case.sh line 19:
export ROOT_PATH=$(pwd)
       ^-------^ SC2155 (warning): Declare and assign separately to avoid masking return values.


In pytest/run_case.sh line 26:
rm -f ${ROOT_PATH}/*.xml | tee
      ^----------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
rm -f "${ROOT_PATH}"/*.xml | tee


In pytest/run_case.sh line 27:
rm -f ${ROOT_PATH}/*.html | tee
      ^----------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
rm -f "${ROOT_PATH}"/*.html | tee


In pytest/run_case.sh line 28:
rm -rf ${REPORT_PATH}
       ^------------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
rm -rf "${REPORT_PATH}"


In pytest/run_case.sh line 29:
mkdir -p ${REPORT_PATH}
         ^------------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
mkdir -p "${REPORT_PATH}"


In pytest/run_case.sh line 31:
cd ${SYS_PATH}
^------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
   ^---------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
cd "${SYS_PATH}" || exit


In pytest/run_case.sh line 57:
    cd ${ENV_PATH}
    ^------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
       ^---------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
    cd "${ENV_PATH}" || exit


In pytest/run_case.sh line 67:
    ls ${case_file}
       ^----------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
    ls "${case_file}"


In pytest/run_case.sh line 94:
    cd ${SYS_PATH}
    ^------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
       ^---------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
    cd "${SYS_PATH}" || exit


In pytest/run_case.sh line 131:
    cd "${QE_PATH}"
    ^-------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${QE_PATH}" || exit


In pytest/run_case.sh line 136:
    cd "${QE_BUG_PATH}"
    ^-----------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${QE_BUG_PATH}" || exit


In pytest/run_case.sh line 193:
    cd "${SYS_PATH}"
    ^--------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}" || exit


In pytest/run_case.sh line 200:
    cd "${SYS_PATH}"/test_sys_routine_load/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_routine_load/ || exit


In pytest/run_case.sh line 207:
    cd "${SYS_PATH}"/test_sys_broker_load/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_broker_load/ || exit


In pytest/run_case.sh line 214:
    cd "${SYS_PATH}"/test_sys_backup_restore
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_backup_restore || exit


In pytest/run_case.sh line 224:
    cd "${SYS_PATH}"/test_sys_stream_load/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_stream_load/ || exit


In pytest/run_case.sh line 241:
    cd "${SYS_PATH}"/test_sys_alter_for_uniq/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_alter_for_uniq/ || exit


In pytest/run_case.sh line 258:
    cd "${SYS_PATH}"/test_sys_alter_for_duplicate/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_alter_for_duplicate/ || exit


In pytest/run_case.sh line 275:
    cd "${SYS_PATH}"/test_sys_alter_for_aggregate/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_alter_for_aggregate/ || exit


In pytest/run_case.sh line 283:
    cd "${SYS_PATH}"/test_sys_bitmap/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_bitmap/ || exit


In pytest/run_case.sh line 291:
    cd "${SYS_PATH}"/test_sys_export/
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_export/ || exit


In pytest/run_case.sh line 296:
    cd "${SYS_PATH}"/test_sys_string
    ^-- SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_string || exit


In pytest/run_case.sh line 304:
    cd "${SYS_PATH}"/test_sys_array
    ^-----------------------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}"/test_sys_array || exit


In pytest/run_restart_case.sh line 19:
export ROOT_PATH=$(pwd)
       ^-------^ SC2155 (warning): Declare and assign separately to avoid masking return values.


In pytest/run_restart_case.sh line 42:
cd "${SYS_PATH}"/test_scene
^-------------------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
cd "${SYS_PATH}"/test_scene || exit


In pytest/run_restart_case.sh line 48:
    cd "${SYS_PATH}"
    ^--------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
    cd "${SYS_PATH}" || exit


In pytest/run_restart_case.sh line 55:
cd "${SYS_PATH}"
^--------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
cd "${SYS_PATH}" || exit

For more information:
  https://www.shellcheck.net/wiki/SC2154 -- FE_DB is referenced but not assig...
  https://www.shellcheck.net/wiki/SC2155 -- Declare and assign separately to ...
  https://www.shellcheck.net/wiki/SC2164 -- Use 'cd ... || exit' or 'cd ... |...
----------

You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
  # shellcheck disable=NNNN
above the line that contains the issue, where NNNN is the error code;
3. Add '-e NNNN' to the SHELLCHECK_OPTS setting in your .yml action file.

shfmt errors


'shfmt ' returned error 1 finding the following formatting issues:

----------
--- pytest/deploy/qe/init_data.sh.orig
+++ pytest/deploy/qe/init_data.sh
@@ -1,4 +1,4 @@
-#!/bin/sh	
+#!/bin/sh
 # Licensed to the Apache Software Foundation (ASF) under one
 # or more contributor license agreements.  See the NOTICE file
 # distributed with this work for additional information
@@ -32,4 +32,3 @@
 
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" "${FE_DB}" -e 'create table bigtable(k1 tinyint, k2 smallint, k3 int, k4 bigint, k5 decimal(9,3), k6 char(5), k10 date, k11 datetime, k7 varchar(20), k8 double max, k9 float sum) engine=olap distributed by hash(k1) buckets 5 properties("storage_type"="column")'
 curl --location-trusted -T baseall.txt -u "${FE_USER}":"${FE_PASSWORD}" http://"${FE_HOST}":"${FE_WEB_PORT}"/api/"${FE_DB}"/bigtable/_stream_load
-
--- pytest/deploy/qe/init_data_broker.sh.orig
+++ pytest/deploy/qe/init_data_broker.sh
@@ -20,7 +20,6 @@
 # hdfs_data_path='hdfs://xxxxxx'
 # broker_property="WITH BROKER 'hdfs' ('username'='root', 'password'='')"
 
-
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "DROP DATABASE IF EXISTS ${FE_DB}"
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "create database ${FE_DB}"
 
@@ -29,7 +28,6 @@
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "create table ${FE_DB}.bigtable(k1 tinyint, k2 smallint, k3 int, k4 bigint, k5 decimal(9,3), k6 char(5), k10 date, k11 datetime, k7 varchar(20), k8 double max, k9 float sum) engine=olap distributed by hash(k1) buckets 5 properties('storage_type'='column')"
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "CREATE TABLE ${FE_DB}.all_type_table (tinyint_key tinyint(4) NULL, smallint_key smallint(6) NULL, int_key int(11) NOT NULL, bigint_key bigint(20) NULL, char_50_key char(50) NULL, character_key varchar(500) NOT NULL, char_key char(1) NULL, character_most_key varchar(65533) NULL, decimal_key decimal(20, 6) NULL, decimal_most_key decimal(27, 9) NULL, date_key date NULL, datetime_key datetime NULL, tinyint_value tinyint(4) SUM NULL, smallint_value smallint(6) SUM NULL, int_value int(11) SUM NULL, largeint_value largeint SUM NULL, char_50_value char(50) REPLACE NULL, character_value varchar(500) REPLACE NULL, char_value char(1) REPLACE NULL, character_most_value varchar(65533) REPLACE NULL, decimal_value decimal(20, 6) SUM NULL, decimal_most_value decimal(27, 9) SUM NULL, date_value_replace date REPLACE NULL, date_value_max date REPLACE NULL, date_value_min date REPLACE NULL, datetime_value_replace datetime REPLACE NULL, datetime_value_max datetime REPLACE NULL, datetime_value_min datetime REPLACE NULL, float_value float SUM NULL, double_value double SUM NULL) ENGINE=OLAP DISTRIBUTED BY HASH(tinyint_key) BUCKETS 5 PROPERTIES ('storage_type' = 'COLUMN')"
 
-
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_1 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE baseall COLUMNS TERMINATED BY '${separator}') ${broker_property}"
 
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_2 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE test COLUMNS TERMINATED BY '${separator}') ${broker_property}"
--- pytest/deploy/qe/init_data_list.sh.orig
+++ pytest/deploy/qe/init_data_list.sh
@@ -37,5 +37,4 @@
 
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL ${FE_DB}.label_4 (DATA INFILE('${hdfs_data_path}/qe/baseall.txt') INTO TABLE bigtable COLUMNS TERMINATED BY '${separator}') ${broker_property}"
 
-
 sleep 20
--- pytest/deploy/qe/init_data_stream.sh.orig
+++ pytest/deploy/qe/init_data_stream.sh
@@ -1,4 +1,4 @@
-#!/bin/sh	
+#!/bin/sh
 # Licensed to the Apache Software Foundation (ASF) under one
 # or more contributor license agreements.  See the NOTICE file
 # distributed with this work for additional information
@@ -32,4 +32,3 @@
 
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" "${FE_DB}" -e 'create table bigtable(k1 tinyint, k2 smallint, k3 int, k4 bigint, k5 decimal(9,3), k6 char(5), k10 date, k11 datetime, k7 varchar(20), k8 double max, k9 float sum) engine=olap distributed by hash(k1) buckets 5 properties("storage_type"="column")'
 curl --location-trusted -T baseall.txt -u "${FE_USER}":"${FE_PASSWORD}" http://"${FE_HOST}":"${FE_WEB_PORT}"/api/"${FE_DB}"/bigtable/_stream_load
-
--- pytest/download_data.sh.orig
+++ pytest/download_data.sh
@@ -16,11 +16,13 @@
 # specific language governing permissions and limitations
 # under the License.
 
-
 set -eo pipefail
 
 PYTEST_HOME=$(dirname "$0")
-PYTEST_HOME=$(cd "${PYTEST_HOME}"; pwd)
+PYTEST_HOME=$(
+    cd "${PYTEST_HOME}"
+    pwd
+)
 
 export PYTEST_HOME
 
@@ -39,7 +41,7 @@
     wget -O "${HDFS_DATA}" https://palo-qa.cdn.bcebos.com/pytest_data/hdfs_data.tar.gz
     rm -rf hdfs
     mkdir -p "${PYTEST_HOME}"/hdfs/
-    tar xzf "${HDFS_DATA}" -C "${PYTEST_HOME}"/hdfs/ 
+    tar xzf "${HDFS_DATA}" -C "${PYTEST_HOME}"/hdfs/
     echo "hdfs data download finished, need to put to hdfs"
 fi
 
@@ -69,6 +71,3 @@
     fi
     echo "local data download finished"
 fi
-
-
-
--- pytest/qe/palo2/data/init_lateral_view.sh.orig
+++ pytest/qe/palo2/data/init_lateral_view.sh
@@ -74,7 +74,7 @@
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e 'create table test_lateral_view.test_multi_partition_bucket(k1 int, k2 int, k3 int, k4 varchar(500), k5 varchar(500), k6 varchar(500), k7 varchar(500), k8 varchar(500), k9 bitmap bitmap_union) PARTITION BY RANGE(k1,k2) (partition p1 values less than ("-32768","-8"), partition p2 values less than ("0","0"),partition p3 values less than ("32768","8"),partition p4 values less than ("65536","18")) DISTRIBUTED BY HASH(k1,k2,k3,k4,k5,k6,k7,k8)'
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL test_lateral_view.label_18 (DATA INFILE('${hdfs_path}/lateral_view_data') INTO TABLE test_multi_partition_bucket (k1,k2,k3,k4,k5,k6,k7,k8) set (k9=bitmap_from_string(k8))) ${broker_property}"
 
-mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e  'create table test_lateral_view.test_explode_split_type(k1 int, k2 char(255), k3 varchar(500), k4 string) DISTRIBUTED BY HASH(k1)'
+mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e 'create table test_lateral_view.test_explode_split_type(k1 int, k2 char(255), k3 varchar(500), k4 string) DISTRIBUTED BY HASH(k1)'
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL test_lateral_view.label_19 (DATA INFILE('${hdfs_path}/lateral_view_data') INTO TABLE test_explode_split_type (k1,v2,v3,v4,v5,v6,v7,v8) set (k2=v4,k3=v4,k4=v4)) ${broker_property}"
 
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e 'create table test_lateral_view.test_explode_split_type_check(k1 int, k2 varchar(50)) DISTRIBUTED BY HASH(k1)'
@@ -85,4 +85,3 @@
 
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e 'create table test_lateral_view.lateral_view_multi_check(k1 int, k2 int, k3 int, k4 varchar(200), k5 bigint, k6 bigint, k7 double) DISTRIBUTED BY HASH(k1)'
 mysql -h "${FE_HOST}" -P"${FE_QUERY_PORT}" -u"${FE_USER}" -p"${FE_PASSWORD}" -e "LOAD LABEL test_lateral_view.label_22 (DATA INFILE('${hdfs_path}/lateral_view_multi_check') INTO TABLE lateral_view_multi_check (k1,k2,k3,k4,k5,k6,k7)) ${broker_property}"
-
--- pytest/run_case.sh.orig
+++ pytest/run_case.sh
@@ -46,13 +46,11 @@
 exec 3<>/tmp/fd1
 rm -rf /tmp/fd1
 
-for ((i=1;i<=PROCESS_NUM;i++))
-do
-        echo >&3
+for ((i = 1; i <= PROCESS_NUM; i++)); do
+    echo >&3
 done
 
-function check_cluster ()
-{
+function check_cluster() {
     # check if fe or be dead and start it
     cd ${ENV_PATH}
     python node_op.py -c '''
@@ -61,42 +59,37 @@
     '''
 }
 
-function pytest_execute ()
-{
+function pytest_execute() {
     case_file=$1
     ls ${case_file}
-    pytest -sv --junit-xml="${REPORT_PATH}"/"${case_file%.py}".xml --html="${REPORT_PATH}"/"${case_file%.py}".html  "${case_file}" --tb=native 2>&1 | tee "${case_file%.py}".log
+    pytest -sv --junit-xml="${REPORT_PATH}"/"${case_file%.py}".xml --html="${REPORT_PATH}"/"${case_file%.py}".html "${case_file}" --tb=native 2>&1 | tee "${case_file%.py}".log
     sleep 1
 }
 
-function case_execute ()
-{
-    for case_file in "$@"
-    do
+function case_execute() {
+    for case_file in "$@"; do
         read -r -u3
         {
             echo "run ${case_file}"
             pytest_execute "${case_file}"
             if [[ "${RESTART_BE_AUTO}" == true ]]; then
                 check_cluster
-            fi     
+            fi
             echo >&3
-        }&
+        } &
     done
 
 }
 
 # 执行时间较长，最先执行
-function long_time_case ()
-{
+function long_time_case() {
     case_files='test_sys_partition_complex_without_restart_be.py 
                 test_sys_storage_medium.py test_sys_delete_load.py'
     cd ${SYS_PATH}
     case_execute "${case_files}"
 }
 
-function query_case ()
-{
+function query_case() {
     # query case
     case_files='test_query_predicates.py 
                 test_query_others.py
@@ -137,8 +130,7 @@
     case_execute "${case_files}"
 }
 
-function sys_case ()
-{
+function sys_case() {
     # sys case
     case_files='test_sys_verify.py
                 test_sys_bloom_filter_a.py
@@ -290,7 +282,7 @@
                 test_select_into_query.py'
     cd "${SYS_PATH}"/test_sys_export/
     case_execute "${case_files}"
-   
+
     # sys/test_sys_string
     case_files='test_sys_string_basic.py'
     cd "${SYS_PATH}"/test_sys_string
--- pytest/run_restart_case.sh.orig
+++ pytest/run_restart_case.sh
@@ -29,11 +29,10 @@
     testsuite=normal
 fi
 
-function pytest_execute()
-{
+function pytest_execute() {
     case_file=$1
     ls "${case_file}"
-    pytest -sv --junit-xml="${REPORT_PATH}"/"${case_file%.py}".xml --html="${REPORT_PATH}"/"${case_file%.py}".html  "${case_file}" --tb=native 2>&1 | tee "${case_file%.py}".log
+    pytest -sv --junit-xml="${REPORT_PATH}"/"${case_file%.py}".xml --html="${REPORT_PATH}"/"${case_file%.py}".html "${case_file}" --tb=native 2>&1 | tee "${case_file%.py}".log
     sleep 1
 }
 
@@ -43,8 +42,7 @@
 pytest_execute "${case}"
 
 cases='test_sys_update_restart.py test_sys_partition_complex_with_restart_be.py test_sys_restart.py test_sys_binlog_restart.py test_sys_resource_tag.py'
-for i in ${cases}
-do
+for i in ${cases}; do
     cd "${SYS_PATH}"
     echo "${i}"
     pytest_execute "${i}"
----------

You can reformat the above files to meet shfmt's requirements by typing:

  shfmt  -w filename

Gabriel39 and others added 30 commits November 3, 2023 10:35

[keyword](decimalv2) Add DecimalV2 keyword apache#26283 (apache#26319)

73ff453

[fix](spark-load)fix-Unique-key-with-MOR-by-sparkload apache#26383 (a…

a415936

…pache#26414)

[fix](nereids)fix bug of select mv in nereids apache#26235 (apache#26415

db460dd

)

[improvement](show trash) Fix be restart slow when too many trash files

c84d82b

apache#26147 (apache#26417)

[fix](planner)should keep at least one slot materialized in agg node a…

3199379

…pache#26116 (apache#26419)

[fix](multi-catalog)add the FAQ for Aliyun DLF and add the fs.xx.impl…

608fd1d

… check apache#25594 (apache#26422)

[coverage](pipeline) Remove unless code and add call method for cover…

48d1fef

…age apache#25552 (apache#26423) Remove unless code and add call method for coverage

[Fix](statistics)Fix analyze min max sql syntax error. apache#26240 (a…

a8e74d6

…pache#26443) backport apache#26240

[fix](auditlog) fix without lock in QueryStatisticsRecvr find (apache…

7ef96b2

…#26441)

[fix](invert index) Fix the timing error when opening the searcher ap…

9ccfd18

…ache#26401 (apache#26472)

[fix](nereids)only enable colocate scan for one phase global parttion…

19998ce

… topn in some condition apache#26473 (apache#26481)

[branch-2.0](cherry-pick) Add more indexed column reader be unit test a…

d92b9c8

…pache#25652 (apache#26430)

[fix](case) rm non-visiable charactor null in out file (apache#26540)

bc3b612

[test](regression) Add more regression test for FE (apache#26539)

c8bc6d0

[test](coverage) Improve test coverage for runtime filter (apache#26314…

0c03172

…) (apache#26547)

[fix](Nereids) could not run query with repeat node in cte (apache#26330

2842d05

) (apache#26531) pick from master PR: apache#26330 commit id: a89477e ExpressionDeepCopier not process VirtualReference, so we generate inline plan with mistake.

[opt](Nereids) remove Nondeterministic trait from date related functi…

9f76dde

…ons (apache#26444) (apache#26568)

change version to 2.0.3-rc03dev

9b8d3b9

[fix](regression-test) Fix regiressin test syncer suit use master fe …

e8ab033

…directly (apache#26456) (apache#26583) Signed-off-by: Jack Drogon <jack.xsuperman@gmail.com>

Revert "[improvement](scanner_schedule) reduce memory consumption of …

ea676de

…scanner apache#24199 (apache#25547)" (apache#26613) This reverts commit 9a19581 to investigate ANALYZE TABLE WITH SYNC problem

[enhancement](Nereids): add LOG info to show the phase of NereidsPlan…

1c3e93b

…ner. (apache#26542)

[opt](regression test) Add string-like column order by test apache#26379

5912bba

(apache#26533)

[branch-2.0](JdbcCatalog) fix that the predicate column name does not…

c355adb

… have back quote when querying the JDBC appearance (apache#26479) (apache#26560) master pr: apache#26479

[fix](prepare statement) Not supported such prepared statement if pre…

0d83327

…pare a forward master sql (apache#26512) (apache#26638)

[Pick-2.0](regression) add failure injection in inverted index writer a…

00143f7

…pache#26121 (apache#26376)

BePPPower and others added 6 commits November 22, 2023 00:20

[Branch2.0](Export) add show export regression testes (apache#27330)

7f3f2d6

[fix](stats) Fix thread leaks when doing checkpoint apache#27334 apac…

f80c9b5

…he#27335

[fix](stats) Fix creating too many tasks on new env (apache#27362)

ffd7521

[fix](build index) fix core when build index for a new column which w…

6c3b352

…ithout data (apache#27276)

change version to 2.0.3-rc04 (apache#27392)

87c4d1c

Merge remote-tracking branch 'upstream-apache/branch-2.0' into branch…

450c1e1

…-2.0-var

eldenmoon closed this Nov 22, 2023

github-actions bot added area/load Issues or PRs related to all kinds of load area/nereids area/pipeline area/planner Issues or PRs related to the query planner kind/docs Categorizes issue or PR as related to documentation. labels Nov 22, 2023

eldenmoon reopened this Nov 22, 2023

github-actions bot reviewed Nov 22, 2023

View reviewed changes

fix merge

f3f54ca

eldenmoon force-pushed the branch-2.0-var-merge branch from bc48921 to f3f54ca Compare November 22, 2023 13:25

update clucene

15dd6cf

eldenmoon merged commit e8abd5c into apache:branch-2.0-var Nov 23, 2023
9 of 15 checks passed

eldenmoon deleted the branch-2.0-var-merge branch November 23, 2023 13:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Branch 2.0 var merge #27422

Branch 2.0 var merge #27422

eldenmoon commented Nov 22, 2023

github-actions bot commented Nov 22, 2023

eldenmoon commented Nov 22, 2023

github-actions bot commented Nov 22, 2023

github-actions bot left a comment

github-actions bot Nov 22, 2023

github-actions bot Nov 22, 2023

github-actions bot Nov 22, 2023

eldenmoon commented Nov 22, 2023

eldenmoon commented Nov 22, 2023

eldenmoon commented Nov 22, 2023

github-actions bot commented Nov 22, 2023

eldenmoon commented Nov 23, 2023

hello-stephen commented Nov 23, 2023

eldenmoon commented Nov 23, 2023

github-actions bot commented Nov 23, 2023

Branch 2.0 var merge #27422

Branch 2.0 var merge #27422

Conversation

eldenmoon commented Nov 22, 2023

Proposed changes

Further comments

github-actions bot commented Nov 22, 2023

sh-checker report

eldenmoon commented Nov 22, 2023

github-actions bot commented Nov 22, 2023

sh-checker report

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot Nov 22, 2023

Choose a reason for hiding this comment

github-actions bot Nov 22, 2023

Choose a reason for hiding this comment

github-actions bot Nov 22, 2023

Choose a reason for hiding this comment

eldenmoon commented Nov 22, 2023

eldenmoon commented Nov 22, 2023

eldenmoon commented Nov 22, 2023

github-actions bot commented Nov 22, 2023

sh-checker report

eldenmoon commented Nov 23, 2023

hello-stephen commented Nov 23, 2023

eldenmoon commented Nov 23, 2023

github-actions bot commented Nov 23, 2023

sh-checker report

`sh-checker report`

`sh-checker report`

`sh-checker report`

`sh-checker report`