日志解析截断异常

[yangshike]

前面1条日志将后面1条的时间字段 截取了一半。导致后面的时间格式错误：

异常日志时间格式："time":"3-20 17:43:39.002"

日志解析 多行配置：
Multiline:
StartPattern: [?\d+-\d+-\d+\s\d+:\d+:\d+.\d+]?\s.*
字段解析配置：
Regex: [?(\d+-\S* \d+:\d+:\d+.\d+)]? (\w+)?\s?[(.?)] ([A-Z]+) (.)
ilogtail 日志：
[2024-03-20 09:36:12.014617] [info] [48] /src/core/reader/LogFileReader.cpp:1258 close file succeeded, project: logstore: config:regex_log_bx log reader queue name:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1228/fs/cloud-core-product-service/core-product-service-full.log real file path:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1228/fs/cloud-core-product-service/core-product-service-full.log file device:64769 file inode:6816113 file signature:6630514858024706238 file signature size:1024 file size:65197 last file position:65197 reader id:113380096
[2024-03-20 09:37:05.674729] [info] [11] /src/core/monitor/LogtaiMetric.cpp:259 writeMetricsTotal:4 writeMetricsDeleteTotal:0 metricsSnapshotTotal:4
[2024-03-20 09:37:26.998316] [info] [48] /src/core/reader/LogFileReader.cpp:1129 open file succeeded, project: logstore: config:regex_log_bx log reader queue name:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1346/fs/cloud-job/job-full.log real file path:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1346/fs/cloud-job/job-full.log file device:64769 file inode:6947092 file signature:4927638916493799451 file signature size:1024 last file position:4756211 reader id:113377984
[2024-03-20 09:38:05.679937] [info] [11] /src/core/monitor/LogtaiMetric.cpp:259 writeMetricsTotal:4 writeMetricsDeleteTotal:0 metricsSnapshotTotal:4
[2024-03-20 09:39:05.684852] [info] [11] /src/core/monitor/LogtaiMetric.cpp:259 writeMetricsTotal:4 writeMetricsDeleteTotal:0 metricsSnapshotTotal:4
[2024-03-20 09:39:12.012459] [info] [48] /src/core/reader/LogFileReader.cpp:1198 close the file:current log file has not been updated for some time and has been read project: logstore: config:regex_log_bx log reader queue name:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1346/fs/cloud-job/job-full.log file device:64769 file inode:6947092 file signature:4927638916493799451 file signature size:1024 file size:4757561 last file position:4757561
[2024-03-20 09:39:12.012517] [info] [48] /src/core/reader/LogFileReader.cpp:1225 update the real file path of the log reader during closing, project: logstore: config:regex_log_bx log reader queue name:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1346/fs/cloud-job/job-full.log file device:64769 file inode:6947092 file signature:4927638916493799451 file signature size:1024 original file path:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1346/fs/cloud-job/job-full.log new file path:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1346/fs/cloud-job/job-full.log
[2024-03-20 09:39:12.012528] [info] [48] /src/core/reader/LogFileReader.cpp:1258 close file succeeded, project: logstore: config:regex_log_bx log reader queue name:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1346/fs/cloud-job/job-full.log real file path:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1346/fs/cloud-job/job-full.log file device:64769 file inode:6947092 file signature:4927638916493799451 file signature size:1024 file size:4757561 last file position:4757561 reader id:113377984
[2024-03-20 09:40:05.690071] [info] [11] /src/core/monitor/LogtaiMetric.cpp:259 writeMetricsTotal:4 writeMetricsDeleteTotal:0 metricsSnapshotTotal:4
[2024-03-20 09:40:51.163455] [info] [48] /src/core/reader/LogFileReader.cpp:1129 open file succeeded, project: logstore: config:regex_log_bx log reader queue name:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1346/fs/cloud-job/job-full.log real file path:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1346/fs/cloud-job/job-full.log file device:64769 file inode:6947092 file signature:4927638916493799451 file signature size:1024 last file position:4757561 reader id:113377984
[2024-03-20 09:41:05.695322] [info] [11] /src/core/monitor/LogtaiMetric.cpp:259 writeMetricsTotal:4 writeMetricsDeleteTotal:0 metricsSnapshotTotal:4
[2024-03-20 09:42:05.700424] [info] [11] /src/core/monitor/LogtaiMetric.cpp:259 writeMetricsTotal:4 writeMetricsDeleteTotal:0 metricsSnapshotTotal:4
[2024-03-20 09:42:12.013863] [info] [48] /src/core/reader/LogFileReader.cpp:1198 close the file:current log file has not been updated for some time and has been read project: logstore: config:regex_log_bx log reader queue name:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1346/fs/cloud-job/job-full.log file device:64769 file inode:6947092 file signature:4927638916493799451 file signature size:1024 file size:4758911 last file position:4758911
[2024-03-20 09:42:12.013904] [info] [48] /src/core/reader/LogFileReader.cpp:1225 update the real file path of the log reader during closing, project: logstore: config:regex_log_bx log reader queue name:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1346/fs/cloud-job/job-full.log file device:64769 file inode:6947092 file signature:4927638916493799451 file signature size:1024 original file path:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1346/fs/cloud-job/job-full.log new file path:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1346/fs/cloud-job/job-full.log
[2024-03-20 09:42:12.013915] [info] [48] /src/core/reader/LogFileReader.cpp:1258 close file succeeded, project: logstore: config:regex_log_bx log reader queue name:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1346/fs/cloud-job/job-full.log real file path:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1346/fs/cloud-job/job-full.log file device:64769 file inode:6947092 file signature:4927638916493799451 file signature size:1024 file size:4758911 last file position:4758911 reader id:113377984
[2024-03-20 09:43:05.705537] [info] [11] /src/core/monitor/LogtaiMetric.cpp:259 writeMetricsTotal:4 writeMetricsDeleteTotal:0 metricsSnapshotTotal:4
[2024-03-20 09:44:05.710717] [info] [11] /src/core/monitor/LogtaiMetric.cpp:259 writeMetricsTotal:4 writeMetricsDeleteTotal:0 metricsSnapshotTotal:4
[2024-03-20 09:44:25.712529] [info] [11] /src/core/controller/EventDispatcher.cpp:990 checkpoint dump:starts
[2024-03-20 09:44:25.712545] [info] [11] /src/core/file_server/FileServer.cpp:62 file server pause:starts
[2024-03-20 09:44:25.712549] [info] [11] /src/core/polling/PollingDirFile.cpp:88 polling discovery pause:starts
[2024-03-20 09:44:25.712552] [info] [11] /src/core/polling/PollingDirFile.cpp:91 polling discovery pause:succeeded
[2024-03-20 09:44:25.712555] [info] [11] /src/core/polling/PollingModify.cpp:73 polling modify pause:starts
[2024-03-20 09:44:25.712557] [info] [11] /src/core/polling/PollingModify.cpp:76 polling modify pause:succeeded
[2024-03-20 09:44:25.712559] [info] [11] /src/core/event_handler/LogInput.cpp:101 event handle daemon pause:starts
[2024-03-20 09:44:25.714317] [info] [11] /src/core/event_handler/LogInput.cpp:104 event handle daemon pause:succeeded
[2024-03-20 09:44:25.714339] [info] [11] /src/core/file_server/FileServer.cpp:75 file server pause:succeeded cost:2ms
[2024-03-20 09:44:25.714398] [info] [11] /src/core/controller/EventDispatcher.cpp:977 save log reader status:succeeded
[2024-03-20 09:44:25.715568] [info] [11] /src/core/file_server/FileServer.cpp:85 file server resume:starts
[2024-03-20 09:44:25.715912] [info] [11] /src/core/file_server/FileServer.cpp:87 watch dirs:succeeded
[2024-03-20 09:44:25.715926] [info] [11] /src/core/event_handler/LogInput.cpp:94 event handle daemon resume:starts
[2024-03-20 09:44:25.715938] [info] [11] /src/core/event_handler/LogInput.cpp:97 event handle daemon resume:succeeded
[2024-03-20 09:44:25.715941] [info] [11] /src/core/polling/PollingModify.cpp:66 polling modify resume:starts
[2024-03-20 09:44:25.715952] [info] [11] /src/core/polling/PollingModify.cpp:69 polling modify resume:succeeded
[2024-03-20 09:44:25.715956] [info] [11] /src/core/polling/PollingDirFile.cpp:95 polling discovery resume:starts
[2024-03-20 09:44:25.715959] [info] [11] /src/core/polling/PollingDirFile.cpp:98 polling discovery resume:succeeded
[2024-03-20 09:44:25.715963] [info] [11] /src/core/file_server/FileServer.cpp:96 file server resume:succeeded
[2024-03-20 09:44:25.715966] [info] [11] /src/core/controller/EventDispatcher.cpp:1001 checkpoint dump:succeeded
[2024-03-20 09:45:05.719387] [info] [11] /src/core/monitor/LogtaiMetric.cpp:259 writeMetricsTotal:4 writeMetricsDeleteTotal:0 metricsSnapshotTotal:4
[2024-03-20 09:45:12.002388] [info] [48] /src/core/reader/LogFileReader.cpp:1198 close the file:current log file has not been updated for some time and has been read project: logstore: config:regex_log_bx log reader queue name:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1434/fs/cloud-user-service/user-service-full.log file device:64769 file inode:6946888 file signature:4163352636354277702 file signature size:1024 file size:11963311 last file position:11963311
[2024-03-20 09:45:12.002430] [info] [48] /src/core/reader/LogFileReader.cpp:1225 update the real file path of the log reader during closing, project: logstore: config:regex_log_bx log reader queue name:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1434/fs/cloud-user-service/user-service-full.log file device:64769 file inode:6946888 file signature:4163352636354277702 file signature size:1024 original file path:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1434/fs/cloud-user-service/user-service-full.log new file path:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1434/fs/cloud-user-service/user-service-full.log
[2024-03-20 09:45:12.002441] [info] [48] /src/core/reader/LogFileReader.cpp:1258 close file succeeded, project: logstore: config:regex_log_bx log reader queue name:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1434/fs/cloud-user-service/user-service-full.log real file path:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1434/fs/cloud-user-service/user-service-full.log file device:64769 file inode:6946888 file signature:4163352636354277702 file signature size:1024 file size:11963311 last file position:11963311 reader id:113384320
[2024-03-20 09:45:12.002454] [info] [48] /src/core/reader/LogFileReader.cpp:1198 close the file:current log file has not been updated for some time and has been read project: logstore: config:regex_log_bx log reader queue name:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1646/fs/cloud-web/web-full.log file device:64769 file inode:8389224 file signature:5536491228763669075 file signature size:1024 file size:48522517 last file position:48522517
[2024-03-20 09:45:12.002466] [info] [48] /src/core/reader/LogFileReader.cpp:1225 update the real file path of the log reader during closing, project: logstore: config:regex_log_bx log reader queue name:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1646/fs/cloud-web/web-full.log file device:64769 file inode:8389224 file signature:5536491228763669075 file signature size:1024 original file path:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1646/fs/cloud-web/web-full.log new file path:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1646/fs/cloud-web/web-full.log
[2024-03-20 09:45:12.002472] [info] [48] /src/core/reader/LogFileReader.cpp:1258 close file succeeded, project: logstore: config:regex_log_bx log reader queue name:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1646/fs/cloud-web/web-full.log real file path:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1646/fs/cloud-web/web-full.log file device:64769 file inode:8389224 file signature:5536491228763669075 file signature size:1024 file size:48522517 last file position:48522517 reader id:113383264
[2024-03-20 09:46:05.724553] [info] [11] /src/core/monitor/LogtaiMetric.cpp:259 writeMetricsTotal:4 writeMetricsDeleteTotal:0 metricsSnapshotTotal:4
[2024-03-20 09:46:14.433822] [info] [48] /src/core/reader/LogFileReader.cpp:1129 open file succeeded, project: logstore: config:regex_log_bx log reader queue name:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1646/fs/cloud-web/web-full.log real file path:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1646/fs/cloud-web/web-full.log file device:64769 file inode:8389224 file signature:5536491228763669075 file signature size:1024 last file position:48522517 reader id:113383264
[2024-03-20 09:46:48.455865] [info] [48] /src/core/reader/LogFileReader.cpp:1129 open file succeeded, project: logstore: config:regex_log_bx log reader queue name:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1434/fs/cloud-user-service/user-service-full.log real file path:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1434/fs/cloud-user-service/user-service-full.log file device:64769 file inode:6946888 file signature:4163352636354277702 file signature size:1024 last file position:11963311 reader id:113384320
[2024-03-20 09:47:05.729708] [info] [11] /src/core/monitor/LogtaiMetric.cpp:259 writeMetricsTotal:4 writeMetricsDeleteTotal:0 metricsSnapshotTotal:4
[2024-03-20 09:48:05.734760] [info] [11] /src/core/monitor/LogtaiMetric.cpp:259 writeMetricsTotal:4 writeMetricsDeleteTotal:0 metricsSnapshotTotal:4
[2024-03-20 09:49:05.739863] [info] [11] /src/core/monitor/LogtaiMetric.cpp:259 writeMetricsTotal:4 writeMetricsDeleteTotal:0 metricsSnapshotTotal:4
[2024-03-20 09:50:05.745099] [info] [11] /src/core/monitor/LogtaiMetric.cpp:259 writeMetricsTotal:4 writeMetricsDeleteTotal:0 metricsSnapshotTotal:4
[2024-03-20 09:51:05.750375] [info] [11] /src/core/monitor/LogtaiMetric.cpp:259 writeMetricsTotal:4 writeMetricsDeleteTotal:0 metricsSnapshotTotal:4
[2024-03-20 09:52:03.702298] [info] [48] /src/core/reader/LogFileReader.cpp:1129 open file succeeded, project: logstore: config:regex_log_bx log reader queue name:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1346/fs/cloud-job/job-full.log real file path:/logtail_host/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1346/fs/cloud-job/job-full.log file device:64769 file inode:6947092 file signature:4927638916493799451 file signature size:1024 last file position:4758911 reader id:113377984
[2024-03-20 09:52:05.755481] [info] [11] /src/core/monitor/LogtaiMetric.cpp:259 writeMetricsTotal:4 writeMetricsDeleteTotal:0 metricsSnapshotTotal:4

[henryzhx8]

2024-03-20 17:43:39.002这条日志有没有上传？

[yangshike]

两条都上传了（因为目前的解析配置Regex: [?(\d+-\S* \d+:\d+:\d+.\d+)]? (\w+)?\s?[(.?)] ([A-Z]+) (.)
。 能解析3-20 17:43:39.002字段），截图是前面时间戳的问题 毫秒位的值+1了
第一条：2024-03-20 17:43:39.002 ........ 2024-0
第二条: 3-20 17:43:39.002 ........
因为第二条的时间字段位置上的时间格式不对，导致后续处理失败