Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support DoPutStatementIngest from Arrow Flight SQL 17.0 #6124

Closed
djanderson opened this issue Jul 25, 2024 · 3 comments · Fixed by #6201
Closed

Support DoPutStatementIngest from Arrow Flight SQL 17.0 #6124

djanderson opened this issue Jul 25, 2024 · 3 comments · Fixed by #6201
Labels
arrow Changes to the arrow crate arrow-flight Changes to the arrow-flight crate enhancement Any new improvement worthy of a entry in the changelog

Comments

@djanderson
Copy link
Contributor

djanderson commented Jul 25, 2024

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

I have a Flight server that I'm working to cut over to Flight SQL. The bulk ingest (equivalent of do_put) has been a challenge.

Describe the solution you'd like

I noticed that the current stable version of the Flight SQL protobuf definition (17.0) defines a new CommandStatementIngest that looks like it would do exactly what I need to do.

I believe we would need to

  • Regen the protobuf definitions for the currently stable 17.0
  • Add a new method to the FlightSqlService trait: do_put_statement_ingest
  • Add a new branch to the match statement to match CommandStatementIngest
  • Update examples
    async fn do_put_statement_ingest(
        &self,
        ticket: CommandStatementIngest,
        request: Request<PeekableFlightDataStream>,
    ) -> Result<DoPutUpdateResult, Status> {
        Err(Status::unimplemented("Implement do_put_statement_ingest"))
    }

Describe alternatives you've considered

Additional context

I'm happy to work on a patch but I'm not sure if this would need to be part of a bigger update to fully support Flight SQL 17.0. I haven't done a diff of the specs to understand all the changes.

@djanderson djanderson added the enhancement Any new improvement worthy of a entry in the changelog label Jul 25, 2024
@alamb
Copy link
Contributor

alamb commented Jul 26, 2024

Thanks @djanderson -- this sounds like a good plan to me, and good timing as well -- we have a release planned in about a month that can take breaking changes so we should be able to get this change in by then

I think you have identified the major pieces. If possible it would be helpful to break it into multiple smaller PRs. Perhaps something like:

  1. Update the protobuf definitions
  2. Update the FligthSql service / add example

Thanks again for the offer -- this sounds great

@alamb
Copy link
Contributor

alamb commented Aug 31, 2024

label_issue.py automatically added labels {'arrow'} from #6133

@alamb alamb added the arrow-flight Changes to the arrow-flight crate label Aug 31, 2024
@alamb
Copy link
Contributor

alamb commented Aug 31, 2024

label_issue.py automatically added labels {'arrow-flight'} from #6133

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate arrow-flight Changes to the arrow-flight crate enhancement Any new improvement worthy of a entry in the changelog
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants