Destination Snowflake: succeeded even with Errors

[marcosmarxm]

Expected Behavior

The sync should fail and not succeded if something is wrong.

Current Behavior

For some cases Snowflake is showing as succeded even with fail conditions.

Logs

If applicable, please upload the logs from the failing operation. For sync jobs, you can download the full logs from the UI by going to the sync attempt page and clicking the download logs button at the top right of the logs display window.

Steps to Reproduce

Severity of the bug for you

Very low / Low / Medium / High / Critical

Airbyte Version

Found in the .env file in the root of the project.

Connector Version (if applicable)

Found in the admin page in the UI in the respective Source or Destination tab.

Additional context

Environment, version, integration...

[marcosmarxm]

Issue #4236 can be duplicated but there I'll request some validation about the sync using Mysql.

From @davin:
@jrhizor and I are seeing instances for snowflake where the sync job actually fails with the following error but the job is still marked as successful.

2021-06-23 03:00:27 INFO 2021-06-23 03:00:27 �[32mINFO�[m i.a.i.d.j.c.s.S3StreamCopier(removeFileAndDropTmpTable):181 - {} - Begin cleaning _airbyte_1624417208755__airbyte_raw_tiny_table tmp table in destination.
2021-06-23 03:00:27 INFO 2021-06-23 03:00:27 �[1;31mERROR�[m i.a.i.d.b.BufferedStreamConsumer(close):203 - {} - on close failed.
2021-06-23 03:00:27 INFO net.snowflake.client.jdbc.SnowflakeSQLException: SQL compilation error:
2021-06-23 03:00:27 INFO Schema 'PORTER_TEST_DB.PUBLIC' does not exist or not authorized.
2021-06-23 03:00:27 INFO at net.snowflake.client.jdbc.SnowflakeUtil.checkErrorAndThrowExceptionSub(SnowflakeUtil.java:124) ~[snowflake-jdbc-3.12.14.jar:3.12.14]
2021-06-23 03:00:27 INFO at net.snowflake.client.jdbc.SnowflakeUtil.checkErrorAndThrowException(SnowflakeUtil.java:64) ~[snowflake-jdbc-3.12.14.jar:3.12.14]
2021-06-23 03:00:27 INFO at net.snowflake.client.core.StmtUtil.pollForOutput(StmtUtil.java:434) ~[snowflake-jdbc-3.12.14.jar:3.12.14]
2021-06-23 03:00:27 INFO at net.snowflake.client.core.StmtUtil.execute(StmtUtil.java:338) ~[snowflake-jdbc-3.12.14.jar:3.12.14]
2021-06-23 03:00:27 INFO at net.snowflake.client.core.SFStatement.executeHelper(SFStatement.java:501) ~[snowflake-jdbc-3.12.14.jar:3.12.14]
2021-06-23 03:00:27 INFO at net.snowflake.client.core.SFStatement.executeQueryInternal(SFStatement.java:229) ~[snowflake-jdbc-3.12.14.jar:3.12.14]
2021-06-23 03:00:27 INFO at net.snowflake.client.core.SFStatement.executeQuery(SFStatement.java:167) ~[snowflake-jdbc-3.12.14.jar:3.12.14]
2021-06-23 03:00:27 INFO at net.snowflake.client.core.SFStatement.execute(SFStatement.java:749) ~[snowflake-jdbc-3.12.14.jar:3.12.14]
2021-06-23 03:00:27 INFO at net.snowflake.client.jdbc.SnowflakeStatementV1.executeInternal(SnowflakeStatementV1.java:292) ~[snowflake-jdbc-3.12.14.jar:3.12.14]
2021-06-23 03:00:27 INFO at net.snowflake.client.jdbc.SnowflakeStatementV1.execute(SnowflakeStatementV1.java:342) ~[snowflake-jdbc-3.12.14.jar:3.12.14]
2021-06-23 03:00:28 INFO at io.airbyte.db.jdbc.JdbcDatabase.lambda$execute$0(JdbcDatabase.java:50) ~[io.airbyte-airbyte-db-0.24.7-alpha.jar:?]
2021-06-23 03:00:28 INFO at io.airbyte.db.jdbc.DefaultJdbcDatabase.execute(DefaultJdbcDatabase.java:61) ~[io.airbyte-airbyte-db-0.24.7-alpha.jar:?]
2021-06-23 03:00:28 INFO at io.airbyte.db.jdbc.JdbcDatabase.execute(JdbcDatabase.java:50) ~[io.airbyte-airbyte-db-0.24.7-alpha.jar:?]

Tyler is having the same issue.
@tylerdelange is possible to share your logs having problem with Snowflake?

[sherifnada]

assigned to me for triaging

[tylerdelange]

We are having the same issue. We are using the S3 staging method for Snowflake. For some reason the COPY command will time out if it runs for 30 mins or more. I discussed this with Snowflake and they said that by default the timeout is 2 days so this must be airbyte canceling the COPY command. Apart from that, if the COPY fails, no data is copied to the tables, but airbyte returns that the sync was successfull.

[sherifnada]

@tylerdelange thanks for the context. A fix is currently in progress.

@irynakruk FYI

[etsybaev]

Hi @marcosmarxm.

Could you please add more a bit more details, please? As far as I see from logs - if something fails on the "check" stage. then this fail is supposed to throw an exception that must fail all sync attempts.
I've tried to reproduce this issue, but whatever I did - the sync job showed as failed if something went wrong (wrong schema, creds, etc)

The issue mentioned above #4236 is for Source meanwhile this one is for the destination

[DoNotPanicUA]

Hi @etsybaev,
I have a bonus for you 😄
There is a bug in the integration tests (merged into master).

[marcosmarxm]

@etsybaev

The issue mentioned above #4236 is for Source meanwhile this one is for the destination

#4236 is using Mysql => Snowflake, is a use case to you try and see if this is actually happening.
Is possible to you create a Mysql source and use Snowflake as destination, then sync first time and after remove the mysql user permission and try to sync again.

[etsybaev]

Hi @marcosmarxm. I assume that in case we would need to fix logic on the source connector side. In the case of the connector - it just receives nothing, so nothing to add, so no fails, please correct me If I'm wrong.
If the issue can be reproduced in that way only and we have another ticket for the source connector - maybe we may close this ticket and start looking at the source connector? what do you think? Thanks

[etsybaev]

Hi @sherifnada, maybe you have some ideas? What do you think about my suggestion above? Thanks

[marcosmarxm]

@etsybaev I'm going to create a more reproducible env for you.

[etsybaev]

Hi @marcosmarxm. Do you have any updates?
Did you try to reproduce it recently? Maybe it's already fixed as part of #4617 (title says fix for source, but there are also changes for destinations included)
Thanks

[marcosmarxm]

@etsybaev I'm going to close this after #4617 if I found the problem I'll reopen, thanks