Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: make sql function in scripts return a list of column vectors #1243

Merged

Conversation

killme2008
Copy link
Contributor

@killme2008 killme2008 commented Mar 26, 2023

I hereby agree to the terms of the GreptimeDB CLA

What's changed and what's your intention?

The current sql function in the coprocessor returns a list of lists such as list[list[vector]]. The query result is a list of RecordBatch and it converts eachRecordBatch into a list of column vectors. It's not friendly to users and batch splitting is not necessary at all for the coprocessor.

So in this PR, I changed the return result type to list[Vector]: a list of column vectors. It concatenates all vectors from each batch into one vector for each column.

For example:

import greptime

greptime.query().sql("select number as a, number as b from numbers limit 10")

It will return a list of two elements:

  • The first one is a column and which is a vector.
  • The second one is b column and which is a vector too.

It's much easier for users to process the query result.

Checklist

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.

Refer to a related PR or issue link (optional)

@killme2008 killme2008 marked this pull request as ready for review March 26, 2023 06:20
@killme2008 killme2008 requested review from discord9 and waynexia March 26, 2023 06:21
@codecov
Copy link

codecov bot commented Mar 26, 2023

Codecov Report

Merging #1243 (06e274b) into develop (6f81717) will decrease coverage by 0.50%.
The diff coverage is 47.50%.

@@             Coverage Diff             @@
##           develop    #1243      +/-   ##
===========================================
- Coverage    85.55%   85.06%   -0.50%     
===========================================
  Files          495      495              
  Lines        72771    72790      +19     
===========================================
- Hits         62261    61917     -344     
- Misses       10510    10873     +363     

src/script/src/python/pyo3/vector_impl.rs Outdated Show resolved Hide resolved
src/script/src/python/ffi_types/copr.rs Outdated Show resolved Hide resolved
src/script/src/python/engine.rs Show resolved Hide resolved
@killme2008 killme2008 force-pushed the feature/tweak-script-sql-api branch from d735c77 to 06e274b Compare March 26, 2023 08:15
@killme2008 killme2008 requested a review from waynexia March 26, 2023 08:15
Copy link
Member

@waynexia waynexia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@waynexia waynexia merged commit b4fc8c5 into GreptimeTeam:develop Mar 27, 2023
@killme2008 killme2008 deleted the feature/tweak-script-sql-api branch January 16, 2024 09:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants