Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimization of log collector module #60

Merged
merged 11 commits into from
Dec 20, 2022
Merged

Optimization of log collector module #60

merged 11 commits into from
Dec 20, 2022

Conversation

Davidhua1996
Copy link
Collaborator

What is the purpose of the change

Optimizing the log collector performance.
Related issues: #59.

Brief change log

  • Load balancer strategies for storing stream log.
  • Optimize the performance in sender client.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency):no
  • Anything that affects deployment: no
  • The Core framework, i.e., JobManager, Server.: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? not documented

@jefftlin
Copy link
Collaborator

LGTM！

@jefftlin jefftlin merged commit 4052f2f into WeBankFinTech:dev-0.2.4 Dec 20, 2022
Alexkun pushed a commit to WeDataSphere/Streamis that referenced this pull request Jun 5, 2024
@Davidhua1996
Optimization of log collector module (WeBankFinTech#60)
f7b6291 
* Add keyword filter strategy in log4j/log4j2.

* Use the stream log event timestamp as the enqueue time.

* Init the structure of supporting multiple disks/drivers.

* Load balancer strategies for storing stream log.

* Adjust the default value of configuration.

* Add test unit for job log storage.

* optimize the restful performance.

* Add debug mode in log collector; Add the strategy to discard log cache.

* Fix the problem in closing log4j bucket.

* Optimize the strategies.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Davidhua1996 @jefftlin