[Bug] [hazelcast] hazelcast/REST FAIL

[fanxishu]

Search before asking

• I had searched in the issues and found no similar issues.

What happened

When you call the REST API to retrieve job information after the engine has been running for a while, the task information retrieval fails, even though there is a job running in the engine.
在引擎运行一段时间后调用rest接口获取job信息的时候任务信息读取失败，可是引擎上面是有这个任务且运行中

SeaTunnel Version

2.3.8

SeaTunnel Config

env {
"job.mode"=STREAMING
"job.name"=实时同步
"checkpoint.interval"="360000"
"checkpoint.timeout"="10000"
"savemode.execute.location"=CLUSTER
}
source {
MySQL-CDC {
    format=DEFAULT
    "snapshot.split.size"="8096"
    "snapshot.fetch.size"="1024"
    "incremental.parallelism"="1"
    "connect.timeout.ms"="30000"
    "connect.max-retries"="3"
    "connection.pool.size"="20"
    "chunk-key.even-distribution.factor.lower-bound"="0.05"
    "chunk-key.even-distribution.factor.upper-bound"="100.0"
    "sample-sharding.threshold"="1000"
    "inverse-sampling.rate"="1000"
    "startup.mode"=INITIAL
    "exactly_once"="false"
    "stop.mode"=NEVER
    parallelism="1"
    "result_table_name"=Table15367766757120
    "dag-parsing.mode"=MULTIPLEX
    catalog {
        factory=Mysql
    }
    database-names=[
        "test"
    ]
    table-names=[
        "test.product"
    ]
    base-url="jdbc:mysql://127.0.0.1:3380/test?useSSL=false&useUnicode=true&characterEncoding=utf-8&allowMultiQueries=true&allowPublicKeyRetrieval=true&rewriteBatchedStatements=true"
    username=root
    password="123456"
    server-time-zone=UTC
    driverName="com.mysql.cj.jdbc.Driver"
}
}
transform {
}
sink {
Jdbc {
       "schema_save_mode"="CREATE_SCHEMA_WHEN_NOT_EXIST"
      "data_save_mode"="APPEND_DATA"
       "create_index"="true"
       "connection_check_timeout_sec"=30
       "batch_size"=1000
       "is_exactly_once"="false"
       "max_commit_attempts"=3
       "transaction_timeout_sec"=-1
       "max_retries"="3"
       "auto_commit"="true"
       "support_upsert_by_query_primary_key_exist"="false"
       "multi_table_sink_replica"=1
       "source_table_name"=Table15367766757120
       "generate_sink_sql"=true
       database="eop_conf"
       table=product
       url="jdbc:mysql://127.0.0.1:3380/eop_conf?useUnicode=true&characterEncoding=UTF-8&rewriteBatchedStatements=true"
       password="123456"
       driver="com.mysql.cj.jdbc.Driver"
       user=root
       startup.mode="initial"
}
}

Running Command

911069562043105281  15686439704192_job  RUNNING         2024-11-19 09:45:10.625

Error Exception

http://10.19.11.19:5801/hazelcast/rest/maps/job-info/911069562043105281
{"status":"fail"}
http://10.19.11.19:5803/hazelcast/rest/maps/running-jobs
{"status":"fail"}

2024-11-19 11:06:26,112 INFO  [.c.c.DefaultClassLoaderService] [hz.main.cached.thread-7] - Create classloader for job 911069562043105281 with jars [file:/app01/seatuunel/apache-seatunnel-2.3.8/connectors/connector-cdc-mysql-2.3.8.jar, file:/app01/seatuunel/apache-seatunnel-2.3.8/connectors/connector-jdbc-2.3.8.jar]
2024-11-19 11:06:26,212 INFO  [.c.c.DefaultClassLoaderService] [hz.main.cached.thread-7] - Release classloader for job 911069562043105281 with jars [file:/app01/seatuunel/apache-seatunnel-2.3.8/connectors/connector-cdc-mysql-2.3.8.jar, file:/app01/seatuunel/apache-seatunnel-2.3.8/connectors/connector-jdbc-2.3.8.jar]
2024-11-19 11:06:26,282 WARN  [.Log4j2HttpGetCommandProcessor] [hz.main.cached.thread-7] - [worker-node-1]:5801 [seatunnel] [5.1] An error occurred while handling request HttpCommand [HTTP_GET]{uri='/hazelcast/rest/maps/job-info/911069562043105281'}AbstractTextCommand[HTTP_GET]{requestId=0}
java.lang.NullPointerException: null
        at com.hazelcast.internal.util.JsonUtil.toJsonObject(JsonUtil.java:308) ~[seatunnel-starter.jar:2.3.8]
        at org.apache.seatunnel.engine.server.rest.RestHttpGetCommandProcessor.convertToJson(RestHttpGetCommandProcessor.java:699) ~[seatunnel-starter.jar:2.3.8]
        at org.apache.seatunnel.engine.server.rest.RestHttpGetCommandProcessor.handleJobInfoById(RestHttpGetCommandProcessor.java:373) ~[seatunnel-starter.jar:2.3.8]
        at org.apache.seatunnel.engine.server.rest.RestHttpGetCommandProcessor.handle(RestHttpGetCommandProcessor.java:137) ~[seatunnel-starter.jar:2.3.8]
        at org.apache.seatunnel.engine.server.rest.RestHttpGetCommandProcessor.handle(RestHttpGetCommandProcessor.java:98) ~[seatunnel-starter.jar:2.3.8]
        at com.hazelcast.internal.ascii.TextCommandServiceImpl$CommandExecutor.run(TextCommandServiceImpl.java:402) ~[seatunnel-starter.jar:2.3.8]
        at com.hazelcast.internal.util.executor.CachedExecutorServiceDelegate$Worker.run(CachedExecutorServiceDelegate.java:217) ~[seatunnel-starter.jar:2.3.8]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_191]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_191]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]
        at com.hazelcast.internal.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:76) ~[seatunnel-starter.jar:2.3.8]
        at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102) ~[seatunnel-starter.jar:2.3.8]
2024-11-19 11:06:26,366 WARN  [c.h.i.n.a.RestApiFilter       ] [hz.main.IO.thread-in-1] - [worker-node-1]:5801 [seatunnel] [5.1] No REST group matching URI: /favicon.ico
2024-11-19 11:06:26,422 WARN  [c.h.i.n.a.RestApiFilter       ] [hz.main.IO.thread-in-2] - [worker-node-1]:5801 [seatunnel] [5.1] No REST group matching URI: /favicon.ico
2024-11-19 11:06:26,422 INFO  [c.h.i.s.t.TcpServerConnection ] [hz.main.IO.thread-in-2] - [worker-node-1]:5801 [seatunnel] [5.1] Connection[id=1458, /10.19.11.19:5801->/10.6.11.232:57525, qualifier=null, endpoint=null, remoteUuid=null, alive=false, connectionType=NONE, planeIndex=-1] closed. Reason: Unsupported command received on REST API handler.
2024-11-19 11:06:26,541 WARN  [c.h.i.n.a.RestApiFilter       ] [hz.main.IO.thread-in-0] - [worker-node-1]:5801 [seatunnel] [5.1] No REST group matching URI: /favicon.ico
2024-11-19 11:06:26,541 INFO  [c.h.i.s.t.TcpServerConnection ] [hz.main.IO.thread-in-0] - [worker-node-1]:5801 [seatunnel] [5.1] Connection[id=1459, /10.19.11.19:5801->/10.6.11.232:57526, qualifier=null, endpoint=null, remoteUuid=null, alive=false, connectionType=NONE, planeIndex=-1] closed. Reason: Unsupported command received on REST API handler.

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[liunaijie]

I also meet same issue in 2.3.5
here are some my finding:
#8053 (comment)

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

[github-actions]

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.