Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In order to achieve a relatively unified interface for SPL and other plugins, the interface definition has been optimized. #1254

Merged
merged 5 commits into from
Dec 5, 2023

Conversation

linrunqi08
Copy link
Collaborator

@linrunqi08 linrunqi08 commented Dec 4, 2023

SPL本来设计的Process接口是一进多出,跟其他processor不同,因此SPL完全独立,不实现Processor.h接口。
V2配置这块希望配置统一,SPL不能过分独立,还是希望能够实现Processor.h,因此对Processor.h中process接口进行改造。

  1. 重新定义virtual void Process(std::vector& logGroupList)接口,其他Processor不需要实现,默认实现为取logGroupList[0],调用原来的void Process(PipelineEventGroup& eventGroup)
  2. 相关的上下文代码进行调整

@linrunqi08 linrunqi08 changed the title refine Pipeline processor interface refine pipeline processor interface Dec 4, 2023
@linrunqi08 linrunqi08 changed the title refine pipeline processor interface In order to achieve a relatively unified interface for SPL and other plugins, the interface definition has been optimized. Dec 4, 2023
core/plugin/instance/ProcessorInstance.cpp Show resolved Hide resolved
core/plugin/instance/ProcessorInstance.cpp Outdated Show resolved Hide resolved
core/plugin/interface/Processor.cpp Outdated Show resolved Hide resolved
@linrunqi08 linrunqi08 requested a review from yyuuttaaoo December 5, 2023 07:49
@yyuuttaaoo yyuuttaaoo merged commit 500a83b into feat/pipeline Dec 5, 2023
yyuuttaaoo pushed a commit that referenced this pull request Dec 24, 2023
* complete pipeline definition (#1185)

* add common param extractor

* add input_file

* add input_observer_network

* add flusher_SLS

* add global config module

* improve pipeline def

* polish code

* polish code

* add pipeline implementation

* polish code

* seperate params from input_file

* polish code

* polish code

* add file server

* polish code

* polish code

* polish code

* fix bug

* polish code

* fix bug

* polish code

* refactor code

* refactor event dispatcher

* remove global filter

* refactor app config

* remove shennong and streamlog

* fix bug

* upgrade V1 config to V2

* fix bug

* refactor pipeline management

* add ut

* add ut

* fix e2e

* polish code

* for tmp use

* for tmp use

* polish code

* for tmp use

* for tmp use

* fix

* fix e2e

* temporarily remove observer case

* fix e2e

* fix e2e

* polish code

* Yaml (#3)

* ParseYamlConfig

* polish code

* fix e2e

* remove old config

* cleanup gflags

* fix bug

* polish code

* polish code

* polish code

* fix bug

* polish code

* polish code

* fix code

* polish code

* polish

* fix code

* fix

* fix

* polish code

* fix

* fix

* fix

* polish

* polish

* polish code

* test

* test

* polish

* fix observer

* polish test

* fix code

* polish code

* polish code

* fix bug

* Pipeline processor (#2)

upgrade processor to v2

* polish code

* polish

* polish

* polish code

* polish code

* In order to achieve a relatively unified interface for SPL and other plugins, the interface definition has been optimized. (#1254)

* refine interface

* refine code

* refine code

* refine code

* fix comments

* polish code

* polish

* polish

* polish code

* polish

* fix code

* fix

* fix

* polish code

* add log

* fix

* fix bug

* fix

* fix bug

* polish

* fix

* fix

* fix

* fix

* fix

* fix

* Merge processor changes of main branches (#1259)

* fix: Correct parsing errors in ProcessorParseApsaraNative with large buffer input (#1255)
---------

Co-authored-by: yitao <qiuwencheng.qwc@alibaba-inc.com>

* polish

* add alarm

* polish

* polish

* polish

* polish code

* polish

* polish code

* polish

---------

Co-authored-by: quzard <1191890118@qq.com>
Co-authored-by: linrunqi08 <90741255+linrunqi08@users.noreply.github.com>
Co-authored-by: yitao <qiuwencheng.qwc@alibaba-inc.com>
Co-authored-by: 笃敏 <zhanghaoxiang.zhx@alibaba-inc.com>
@yyuuttaaoo yyuuttaaoo added this to the v2.0 milestone Jan 2, 2024
@yyuuttaaoo yyuuttaaoo added the feature New feature label Jan 11, 2024
@linrunqi08 linrunqi08 deleted the feature/taiye/pipeline branch May 20, 2024 06:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants