Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Pulsar SQL] Pulsar SQL support query big entry data #12448

Merged
merged 6 commits into from
Oct 26, 2021
Merged

[Pulsar SQL] Pulsar SQL support query big entry data #12448

merged 6 commits into from
Oct 26, 2021

Conversation

gaoran10
Copy link
Contributor

@gaoran10 gaoran10 commented Oct 21, 2021 •
edited
Loading

Motivation

Currently, the Pulsar SQL didn't set NettyMaxFrameSizeBytes for the BookKeeper client, so users couldn't use Pulsar SQL query big entry data.

Modifications

Set configuration NettyMaxFrameSizeBytes when generating the BookKeeper client in Pulsar SQL.

setNettyMaxFrameSizeBytes(pulsarConnectorConfig.getMaxMessageSize() + Commands.MESSAGE_SIZE_FRAME_PADDING)

Verifying this change

Add a new test to verify using Pulsar SQL to query big entry data.

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API: (no)
  • The schema: (no)
  • The default values of configurations: (no)
  • The wire protocol: (no)
  • The rest endpoints: (no)
  • The admin cli options: (no)
  • Anything that affects deployment: (no)

Documentation

Check the box below and label this PR (if you have committer privilege).

Need to update docs?

  • doc-required

    (If you need help on updating docs, create a doc issue)

  • no-need-doc

    (Please explain why)

  • doc

    (If this PR contains doc changes)

@eolivelli eolivelli added the doc-not-needed Your PR changes do not impact docs label Oct 21, 2021
@gaoran10 gaoran10 added doc-required Your PR changes impact docs and you will update later. area/sql Pulsar SQL related features and removed doc-not-needed Your PR changes do not impact docs doc-required Your PR changes impact docs and you will update later. labels Oct 21, 2021
@eolivelli eolivelli added the doc Your PR contains doc changes, no matter whether the changes are in markdown or code files. label Oct 21, 2021
@eolivelli
Copy link
Contributor

@gaoran10:Thanks for your contribution. For this PR, do we need to update docs?
(The PR template contains info about doc, which helps others know more about the changes. Can you provide doc-related info in this and future PR descriptions? Thanks)

@eolivelli
Copy link
Contributor

@gaoran10:Thanks for providing doc info!

@codelipenghui codelipenghui merged commit a8f0788 into apache:master Oct 26, 2021
@codelipenghui codelipenghui added this to the 2.10.0 milestone Oct 26, 2021
codelipenghui pushed a commit that referenced this pull request Oct 26, 2021
@gaoran10 @codelipenghui
[Pulsar SQL] Pulsar SQL support query big entry data (#12448)
e7c3bfb 
(cherry picked from commit a8f0788)
@codelipenghui codelipenghui added the cherry-picked/branch-2.8 Archived: 2.8 is end of life label Oct 26, 2021
zeo1995 pushed a commit to zeo1995/pulsar that referenced this pull request Oct 26, 2021
Merge branch 'up/master' into website/branch-2.7.2-chapter-4
fa9166a 
* up/master:
  [C++] Fixed connection read error logging (apache#12492)
  [Pulsar SQL] Pulsar SQL support query big entry data (apache#12448)
  [Java Client] Remove data race in MultiTopicsConsumerImpl to ensure correct message order (apache#12456)
  Allow to have different instances LocalMemoryMetadataStore that share the same state (apache#12390)
  Remove unused ConsumerImpl.isTxnMessage (apache#12472)
@gaoran10 gaoran10 deleted the gaoran/pulsar-sql-netty-max-frame-size branch November 1, 2021 02:53
eolivelli pushed a commit to eolivelli/pulsar that referenced this pull request Nov 29, 2021
@gaoran10 @eolivelli
[Pulsar SQL] Pulsar SQL support query big entry data (apache#12448)
6241645
codelipenghui pushed a commit that referenced this pull request Dec 20, 2021
@gaoran10 @codelipenghui
[Pulsar SQL] Pulsar SQL support query big entry data (#12448)
b892196 
(cherry picked from commit a8f0788)
@codelipenghui codelipenghui added the cherry-picked/branch-2.9 Archived: 2.9 is end of life label Dec 20, 2021
@momo-jun momo-jun mentioned this pull request Mar 28, 2022
Merged
1 task
momo-jun added a commit to momo-jun/pulsar that referenced this pull request Aug 4, 2022
@momo-jun
Merge 2.9.1-specific docs from apache#11988 and 2.9.2-specific docs f…
b034618 
…rom apache#12448, apache#12641, and apache#12786
momo-jun added a commit to momo-jun/pulsar that referenced this pull request Aug 4, 2022
@momo-jun
Merge 2.8.2-specific docs from apache#12641, apache#12504, apache#12448…
890f7c3 
…, and apache#12350
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@codelipenghui codelipenghui codelipenghui approved these changes

@congbobo184 congbobo184 congbobo184 approved these changes

@sijie sijie Awaiting requested review from sijie

@eolivelli eolivelli Awaiting requested review from eolivelli

@hangc0276 hangc0276 Awaiting requested review from hangc0276

@zymap zymap Awaiting requested review from zymap

Assignees

@gaoran10 gaoran10

Labels
area/sql Pulsar SQL related features cherry-picked/branch-2.8 Archived: 2.8 is end of life cherry-picked/branch-2.9 Archived: 2.9 is end of life doc Your PR contains doc changes, no matter whether the changes are in markdown or code files. release/2.8.2 release/2.9.2
Projects
None yet
Milestone
2.10.0
Development

Successfully merging this pull request may close these issues.
4 participants
@gaoran10 @eolivelli @codelipenghui @congbobo184