Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to select different schemas (rather than just public schema) in Postgres integration #1036

Closed
sjlogan opened this issue Nov 20, 2020 · 3 comments

Comments

@sjlogan
Copy link

sjlogan commented Nov 20, 2020

I am attempting to use Airbyte to move data from a postgres db to a warehouse. The data I care about lives in a schema that is not the "public" schema. Airbyte currently only sees the public schema, thus I cannot use the tool to move my data.

When setting up a postgres integration in Airbyte, I would like to be able to see and select from a list of the schemas the postgres user I provided has access to before selecting the tables in that schema that I wish to migrate. Ideally, I would want to be able to select multiple schemas. Of course, this would also mean that Airbyte would have to be intelligent about writing data to their own schemas (or at least prepending schema names to target tables) when writing to the destination.

Stitch.io allows me to select a non-"public" schema as I am setting up a postgres source. If I select multiple schemas, it can prepend schema names to the target tables in the warehouse. They cannot write data to different target schemas in the target warehouse (without setting up multiple integrations), which would be the optimal solution.

Reach out if you have questions or if I can clarify anything.

@sjlogan sjlogan added the type/enhancement New feature or request label Nov 20, 2020
@ChristopheDuong
Copy link
Contributor

Adding conversation happening on slack around this use case:
Screenshot 2021-01-28 at 14 15 20

@cgardens
Copy link
Contributor

cgardens commented Feb 1, 2021

I believe the postgres source now supports multiple schemas. I think we just forgot to update this issue.

@davinchia
Copy link
Contributor

Confirmed we support this today.

Screen Shot 2021-05-19 at 4 02 26 PM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants