SNOW-992383: overwrite=True does not truncate table, although documentation says otherweise

[jan-who]

Python version

3.10.13

Operating system and processor architecture

macOS-14.2-arm64-arm-64bit

Installed packages

asn1crypto==1.5.1
certifi==2023.11.17
cffi==1.16.0
charset-normalizer==3.3.2
cryptography==41.0.7
filelock==3.13.1
idna==3.6
importlib-metadata==7.0.0
jaraco.classes==3.3.0
keyring==24.3.0
more-itertools==10.1.0
numpy==1.26.2
packaging==23.2
pandas==2.1.4
platformdirs==3.11.0
pyarrow==14.0.1
pycparser==2.21
PyJWT==2.8.0
pyOpenSSL==23.3.0
python-dateutil==2.8.2
pytz==2023.3.post1
requests==2.31.0
six==1.16.0
snowflake-connector-python==3.6.0
sortedcontainers==2.4.0
tomlkit==0.12.3
typing_extensions==4.9.0
tzdata==2023.3
urllib3==2.1.0
zipp==3.17.0

What did you do?

`import pandas as pd
import snowflake.connector
from snowflake.connector.pandas_tools import write_pandas

connection = snowflake.connector.connect(...)
df = pd.DataFrame({'col1': [3,4], 'col2': ["abc", "def"]})
write_pandas(conn=connection, df=df, database="RAW", schema="DWD", table_name="TEST", auto_create_table=False, overwrite=True, table_type="transient")

connection.close()`

What did you expect to see?

The table RAW.DWD.TEST was dropped and the new transient table RAW.DWD.<random string> CREATED and then renamed to RAW.DWD.TEST.

From the documentation of write_pandas():

overwrite: When true, and if auto_create_table is true, then it drops the table. Otherwise, it truncates the table. In both cases it will replace the existing contents of the table with that of the passed in Pandas DataFrame.
According to the documentation, the method should truncate my table. That is not the case. If the table exists, the new rows will be appended to the table.

Either the documentation should be updated, or the truncate-functionality should be added.

• overwrite=True, auto_create_table=True: creates new table, drops old table, renames new table to old table
• overwrite=True, auto_create_table=False: creates new table, drops old table, renames new table to old table (same as above!)
• overwrite=False, auto_create_table=True: Appends new data to table
• overwrite=False, auto_create_table=False: Appends new data to table, if it exists. Throws an error otherwise.

overwrite=True, auto_create_table=False does look wrong in its functionality.

Can you set logging to DEBUG and collect the logs?

2023-12-15 13:38:59,426 - MainThread connection.py:790 - cursor() - DEBUG - cursor
2023-12-15 13:38:59,426 - MainThread pandas_tools.py:82 - _do_create_temp_stage() - DEBUG - creating stage with 'CREATE TEMP STAGE /* Python:snowflake.connector.pandas_tools.write_pandas() */ "RAW"."DWD"."cnoxjgjzco" FILE_FORMAT=(TYPE=PARQUET COMPRESSION=gzip BINARY_AS_TEXT=FALSE)'
2023-12-15 13:38:59,426 - MainThread cursor.py:960 - execute() - DEBUG - executing SQL/command
2023-12-15 13:38:59,426 - MainThread cursor.py:816 - _preprocess_pyformat_query() - DEBUG - binding: [CREATE TEMP STAGE /* Python:snowflake.connector.pandas_tools.write_pandas() */ "...] with input=[None], processed=[{}]
2023-12-15 13:38:59,426 - MainThread cursor.py:1028 - execute() - INFO - query: [CREATE TEMP STAGE /* Python:snowflake.connector.pandas_tools.write_pandas() */ "...]
2023-12-15 13:38:59,426 - MainThread connection.py:1542 - _next_sequence_counter() - DEBUG - sequence counter: 1
2023-12-15 13:38:59,427 - MainThread cursor.py:690 - _execute_helper() - DEBUG - Request id: 1a8dd96d-fdf1-4fdb-a08e-140a9666adaa
2023-12-15 13:38:59,427 - MainThread cursor.py:692 - _execute_helper() - DEBUG - running query [CREATE TEMP STAGE /* Python:snowflake.connector.pandas_tools.write_pandas() */ "...]
2023-12-15 13:38:59,427 - MainThread cursor.py:699 - _execute_helper() - DEBUG - is_file_transfer: True
2023-12-15 13:38:59,427 - MainThread connection.py:1205 - cmd_query() - DEBUG - _cmd_query
2023-12-15 13:38:59,427 - MainThread _query_context_cache.py:155 - serialize_to_dict() - DEBUG - serialize_to_dict() called
2023-12-15 13:38:59,427 - MainThread connection.py:1232 - cmd_query() - DEBUG - sql=[CREATE TEMP STAGE /* Python:snowflake.connector.pandas_tools.write_pandas() */ "...], sequence_id=[1], is_file_transfer=[False]
2023-12-15 13:38:59,427 - MainThread network.py:1219 - _use_requests_session() - DEBUG - Session status for SessionPool '**censored**.snowflakecomputing.com', SessionPool 1/1 active sessions
2023-12-15 13:38:59,427 - MainThread network.py:870 - _request_exec_wrapper() - DEBUG - remaining request timeout: N/A ms, retry cnt: 1
2023-12-15 13:38:59,427 - MainThread network.py:852 - add_request_guid() - DEBUG - Request guid: 01371443-05d4-4548-b24d-8e2659d1da7e
2023-12-15 13:38:59,427 - MainThread network.py:1060 - _request_exec() - DEBUG - socket timeout: 60
2023-12-15 13:38:59,593 - MainThread connectionpool.py:474 - _make_request() - DEBUG - https://**censored**.snowflakecomputing.com:443 "POST /queries/v1/query-request?requestId=1a8dd96d-fdf1-4fdb-a08e-140a9666adaa&request_guid=01371443-05d4-4548-b24d-8e2659d1da7e HTTP/1.1" 200 None
2023-12-15 13:38:59,594 - MainThread network.py:1087 - _request_exec() - DEBUG - SUCCESS
2023-12-15 13:38:59,594 - MainThread network.py:1224 - _use_requests_session() - DEBUG - Session status for SessionPool '**censored**.snowflakecomputing.com', SessionPool 0/1 active sessions
2023-12-15 13:38:59,594 - MainThread network.py:740 - _post_request() - DEBUG - ret[code] = None, after post request
2023-12-15 13:38:59,595 - MainThread network.py:766 - _post_request() - DEBUG - Query id: 01b10136-0203-38aa-0001-ab7a0030115a
2023-12-15 13:38:59,595 - MainThread _query_context_cache.py:189 - deserialize_json_dict() - DEBUG - deserialize_json_dict() called: data from server: {'entries': [{'id': 0, 'timestamp': 163192743929263, 'priority': 0}]}
2023-12-15 13:38:59,595 - MainThread _query_context_cache.py:230 - deserialize_json_dict() - DEBUG - deserialize {'id': 0, 'timestamp': 163192743929263, 'priority': 0}
2023-12-15 13:38:59,595 - MainThread _query_context_cache.py:101 - _sync_priority_map() - DEBUG - sync_priority_map called priority_map size = 0, new_priority_map size = 1
2023-12-15 13:38:59,595 - MainThread _query_context_cache.py:127 - trim_cache() - DEBUG - trim_cache() called. treeSet size is 1 and cache capacity is 5
2023-12-15 13:38:59,595 - MainThread _query_context_cache.py:136 - trim_cache() - DEBUG - trim_cache() returns. treeSet size is 1 and cache capacity is 5
2023-12-15 13:38:59,595 - MainThread _query_context_cache.py:269 - deserialize_json_dict() - DEBUG - deserialize_json_dict() returns
2023-12-15 13:38:59,595 - MainThread _query_context_cache.py:274 - log_cache_entries() - DEBUG - Cache Entry: (0, 163192743929263, 0)
2023-12-15 13:38:59,595 - MainThread cursor.py:1035 - execute() - DEBUG - sfqid: 01b10136-0203-38aa-0001-ab7a0030115a
2023-12-15 13:38:59,595 - MainThread cursor.py:1041 - execute() - INFO - query execution done
2023-12-15 13:38:59,595 - MainThread cursor.py:1055 - execute() - DEBUG - SUCCESS
2023-12-15 13:38:59,595 - MainThread cursor.py:1074 - execute() - DEBUG - PUT OR GET: False
2023-12-15 13:38:59,595 - MainThread cursor.py:1191 - _init_result_and_meta() - DEBUG - Query result format: json
2023-12-15 13:38:59,595 - MainThread result_batch.py:487 - _parse() - DEBUG - parsing for result batch id: 1
2023-12-15 13:38:59,595 - MainThread cursor.py:1205 - _init_result_and_meta() - INFO - Number of results in first chunk: 1
2023-12-15 13:38:59,595 - MainThread result_set.py:59 - result_set_iterator() - DEBUG - beginning to schedule result batch downloads
2023-12-15 13:38:59,614 - MainThread pandas_tools.py:343 - write_pandas() - DEBUG - uploading files with 'PUT /* Python:snowflake.connector.pandas_tools.write_pandas() */ 'file:///var/folders/59/ssms5v8s531flb9vk_0p9tgr0000gn/T/tmps30sb20g/file0.txt' @"RAW"."DWD"."cnoxjgjzco" PARALLEL=4'
2023-12-15 13:38:59,614 - MainThread cursor.py:960 - execute() - DEBUG - executing SQL/command
2023-12-15 13:38:59,614 - MainThread cursor.py:816 - _preprocess_pyformat_query() - DEBUG - binding: [PUT /* Python:snowflake.connector.pandas_tools.write_pandas() */ 'file:///var/fo...] with input=[None], processed=[{}]
2023-12-15 13:38:59,614 - MainThread cursor.py:1028 - execute() - INFO - query: [PUT /* Python:snowflake.connector.pandas_tools.write_pandas() */ 'file:///var/fo...]
2023-12-15 13:38:59,614 - MainThread connection.py:1542 - _next_sequence_counter() - DEBUG - sequence counter: 2
2023-12-15 13:38:59,614 - MainThread cursor.py:690 - _execute_helper() - DEBUG - Request id: 1a0cf9db-efd6-466a-82ad-e1fb06c7e695
2023-12-15 13:38:59,614 - MainThread cursor.py:692 - _execute_helper() - DEBUG - running query [PUT /* Python:snowflake.connector.pandas_tools.write_pandas() */ 'file:///var/fo...]
2023-12-15 13:38:59,614 - MainThread cursor.py:699 - _execute_helper() - DEBUG - is_file_transfer: True
2023-12-15 13:38:59,614 - MainThread connection.py:1205 - cmd_query() - DEBUG - _cmd_query
2023-12-15 13:38:59,614 - MainThread _query_context_cache.py:155 - serialize_to_dict() - DEBUG - serialize_to_dict() called
2023-12-15 13:38:59,614 - MainThread _query_context_cache.py:274 - log_cache_entries() - DEBUG - Cache Entry: (0, 163192743929263, 0)
2023-12-15 13:38:59,614 - MainThread _query_context_cache.py:178 - serialize_to_dict() - DEBUG - serialize_to_dict(): data to send to server {'entries': [{'id': 0, 'timestamp': 163192743929263, 'priority': 0, 'context': {}}]}
2023-12-15 13:38:59,614 - MainThread connection.py:1232 - cmd_query() - DEBUG - sql=[PUT /* Python:snowflake.connector.pandas_tools.write_pandas() */ 'file:///var/fo...], sequence_id=[2], is_file_transfer=[True]
2023-12-15 13:38:59,614 - MainThread network.py:1219 - _use_requests_session() - DEBUG - Session status for SessionPool '**censored**.snowflakecomputing.com', SessionPool 1/1 active sessions
2023-12-15 13:38:59,614 - MainThread network.py:870 - _request_exec_wrapper() - DEBUG - remaining request timeout: N/A ms, retry cnt: 1
2023-12-15 13:38:59,614 - MainThread network.py:852 - add_request_guid() - DEBUG - Request guid: c347d1cc-7f18-41dc-a6de-25f150b7fda4
2023-12-15 13:38:59,614 - MainThread network.py:1060 - _request_exec() - DEBUG - socket timeout: 60
2023-12-15 13:38:59,729 - MainThread connectionpool.py:474 - _make_request() - DEBUG - https://**censored**.snowflakecomputing.com:443 "POST /queries/v1/query-request?requestId=1a0cf9db-efd6-466a-82ad-e1fb06c7e695&request_guid=c347d1cc-7f18-41dc-a6de-25f150b7fda4 HTTP/1.1" 200 None
2023-12-15 13:38:59,730 - MainThread network.py:1087 - _request_exec() - DEBUG - SUCCESS
2023-12-15 13:38:59,730 - MainThread network.py:1224 - _use_requests_session() - DEBUG - Session status for SessionPool '**censored**.snowflakecomputing.com', SessionPool 0/1 active sessions
2023-12-15 13:38:59,730 - MainThread network.py:740 - _post_request() - DEBUG - ret[code] = None, after post request
2023-12-15 13:38:59,730 - MainThread network.py:766 - _post_request() - DEBUG - Query id: 01b10136-0203-38a5-0001-ab7a0030314e
2023-12-15 13:38:59,730 - MainThread cursor.py:1035 - execute() - DEBUG - sfqid: 01b10136-0203-38a5-0001-ab7a0030314e
2023-12-15 13:38:59,730 - MainThread cursor.py:1041 - execute() - INFO - query execution done
2023-12-15 13:38:59,730 - MainThread cursor.py:1055 - execute() - DEBUG - SUCCESS
2023-12-15 13:38:59,730 - MainThread cursor.py:1074 - execute() - DEBUG - PUT OR GET: True
2023-12-15 13:38:59,739 - MainThread file_transfer_agent.py:851 - _init_encryption_material() - DEBUG - UPLOAD
2023-12-15 13:38:59,740 - MainThread file_transfer_agent.py:977 - _init_file_metadata() - DEBUG - command type: UPLOAD
2023-12-15 13:38:59,742 - MainThread file_transfer_agent.py:1137 - _process_file_compression_type() - DEBUG - detected the encoding parquet: file=/var/folders/59/ssms5v8s531flb9vk_0p9tgr0000gn/T/tmps30sb20g/file0.txt
2023-12-15 13:38:59,742 - MainThread file_transfer_agent.py:424 - execute() - DEBUG - parallel=[4]
2023-12-15 13:38:59,742 - MainThread file_transfer_agent.py:447 - transfer() - DEBUG - Chunk ThreadPoolExecutor size: 4
2023-12-15 13:38:59,742 - ThreadPoolExecutor-2_0 storage_client.py:187 - preprocess() - DEBUG - Preprocessing /var/folders/59/ssms5v8s531flb9vk_0p9tgr0000gn/T/tmps30sb20g/file0.txt
2023-12-15 13:38:59,742 - ThreadPoolExecutor-2_0 retry.py:351 - from_int() - DEBUG - Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
2023-12-15 13:38:59,742 - ThreadPoolExecutor-2_0 retry.py:351 - from_int() - DEBUG - Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
2023-12-15 13:38:59,742 - ThreadPoolExecutor-2_0 network.py:1219 - _use_requests_session() - DEBUG - Session status for SessionPool 'kwiijjsfcb1stg.blob.core.windows.net', SessionPool 1/1 active sessions
2023-12-15 13:38:59,742 - ThreadPoolExecutor-2_0 storage_client.py:286 - _send_request_with_retry() - DEBUG - storage client request with session <snowflake.connector.vendored.requests.sessions.Session object at 0x155f57250>
2023-12-15 13:38:59,744 - ThreadPoolExecutor-2_0 connectionpool.py:1019 - _new_conn() - DEBUG - Starting new HTTPS connection (1): kwiijjsfcb1stg.blob.core.windows.net:443
2023-12-15 13:38:59,906 - ThreadPoolExecutor-2_0 ssl_wrap_socket.py:79 - ssl_wrap_socket_with_ocsp() - DEBUG - OCSP Mode: FAIL_OPEN, OCSP response cache file name: None
2023-12-15 13:38:59,906 - ThreadPoolExecutor-2_0 ocsp_snowflake.py:530 - reset_ocsp_response_cache_uri() - DEBUG - ocsp_response_cache_uri: file:///Users/jan/Library/Caches/Snowflake/ocsp_response_cache.json
2023-12-15 13:38:59,907 - ThreadPoolExecutor-2_0 ocsp_snowflake.py:533 - reset_ocsp_response_cache_uri() - DEBUG - OCSP_VALIDATION_CACHE size: 201
2023-12-15 13:38:59,907 - ThreadPoolExecutor-2_0 ocsp_snowflake.py:332 - reset_ocsp_dynamic_cache_server_url() - DEBUG - OCSP response cache server is enabled: http://ocsp.snowflakecomputing.com/ocsp_response_cache.json
2023-12-15 13:38:59,907 - ThreadPoolExecutor-2_0 ocsp_snowflake.py:345 - reset_ocsp_dynamic_cache_server_url() - DEBUG - OCSP dynamic cache server RETRY URL: None
2023-12-15 13:38:59,907 - ThreadPoolExecutor-2_0 ocsp_snowflake.py:966 - validate() - DEBUG - validating certificate: kwiijjsfcb1stg.blob.core.windows.net
2023-12-15 13:38:59,907 - ThreadPoolExecutor-2_0 ocsp_asn1crypto.py:385 - extract_certificate_chain() - DEBUG - # of certificates: 2
2023-12-15 13:38:59,907 - ThreadPoolExecutor-2_0 ocsp_asn1crypto.py:390 - extract_certificate_chain() - DEBUG - subject: OrderedDict([('common_name', '*.blob.core.windows.net')]), issuer: OrderedDict([('country_name', 'US'), ('organization_name', 'Microsoft Corporation'), ('common_name', 'Microsoft RSA TLS CA 01')])
2023-12-15 13:38:59,907 - ThreadPoolExecutor-2_0 ocsp_asn1crypto.py:390 - extract_certificate_chain() - DEBUG - subject: OrderedDict([('country_name', 'US'), ('organization_name', 'Microsoft Corporation'), ('common_name', 'Microsoft RSA TLS CA 01')]), issuer: OrderedDict([('country_name', 'IE'), ('organization_name', 'Baltimore'), ('organizational_unit_name', 'CyberTrust'), ('common_name', 'Baltimore CyberTrust Root')])
2023-12-15 13:38:59,909 - ThreadPoolExecutor-2_0 ocsp_asn1crypto.py:413 - create_pair_issuer_subject() - DEBUG - not found issuer_der: OrderedDict([('country_name', 'IE'), ('organization_name', 'Baltimore'), ('organizational_unit_name', 'CyberTrust'), ('common_name', 'Baltimore CyberTrust Root')])
2023-12-15 13:38:59,910 - ThreadPoolExecutor-2_0 ocsp_asn1crypto.py:205 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-12-15 13:38:59,910 - ThreadPoolExecutor-2_0 ocsp_snowflake.py:730 - find_cache() - DEBUG - hit cache for subject: OrderedDict([('common_name', '*.blob.core.windows.net')])
2023-12-15 13:38:59,911 - ThreadPoolExecutor-2_0 ocsp_asn1crypto.py:205 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-05-12 23:58:59+00:00
2023-12-15 13:38:59,911 - ThreadPoolExecutor-2_0 ocsp_snowflake.py:730 - find_cache() - DEBUG - hit cache for subject: OrderedDict([('country_name', 'US'), ('organization_name', 'Microsoft Corporation'), ('common_name', 'Microsoft RSA TLS CA 01')])
2023-12-15 13:38:59,912 - ThreadPoolExecutor-2_0 ocsp_snowflake.py:1023 - _validate() - DEBUG - ok
2023-12-15 13:38:59,975 - ThreadPoolExecutor-2_0 connectionpool.py:474 - _make_request() - DEBUG - https://kwiijjsfcb1stg.blob.core.windows.net:443 "HEAD /stageszza4b74fa3-7879-4751-aba8-ab66e1a896a6/file0.txt?sv=2021-04-10&spr=https&se=2023-12-15T14%3A38%3A59Z&sr=c&sp=rwdl&sig=GqwRGEr0C4QrTy2MSVbbbuq97naxVLd6aP%2BBgvoDiAk%3D HTTP/1.1" 404 0
2023-12-15 13:38:59,975 - ThreadPoolExecutor-2_0 network.py:1224 - _use_requests_session() - DEBUG - Session status for SessionPool 'kwiijjsfcb1stg.blob.core.windows.net', SessionPool 0/1 active sessions
2023-12-15 13:38:59,975 - ThreadPoolExecutor-2_0 storage_client.py:134 - get_digest() - DEBUG - getting digest file=/var/folders/59/ssms5v8s531flb9vk_0p9tgr0000gn/T/tmps30sb20g/file0.txt
2023-12-15 13:38:59,976 - ThreadPoolExecutor-2_0 file_util.py:150 - get_digest_and_size_for_file() - DEBUG - getting digest and size: ahTf84mizSFgYFFfA4bZvQph9dPwUTLVsh46cVFAFsQ=, 2287, file=/var/folders/59/ssms5v8s531flb9vk_0p9tgr0000gn/T/tmps30sb20g/file0.txt
2023-12-15 13:38:59,976 - ThreadPoolExecutor-2_0 storage_client.py:134 - get_digest() - DEBUG - getting digest file=/var/folders/59/ssms5v8s531flb9vk_0p9tgr0000gn/T/tmps30sb20g/file0.txt
2023-12-15 13:38:59,976 - ThreadPoolExecutor-2_0 file_util.py:150 - get_digest_and_size_for_file() - DEBUG - getting digest and size: ahTf84mizSFgYFFfA4bZvQph9dPwUTLVsh46cVFAFsQ=, 2287, file=/var/folders/59/ssms5v8s531flb9vk_0p9tgr0000gn/T/tmps30sb20g/file0.txt
2023-12-15 13:38:59,976 - ThreadPoolExecutor-2_0 storage_client.py:228 - prepare_upload() - DEBUG - Preparing to upload /var/folders/59/ssms5v8s531flb9vk_0p9tgr0000gn/T/tmps30sb20g/file0.txt
2023-12-15 13:38:59,976 - ThreadPoolExecutor-2_0 storage_client.py:150 - encrypt() - DEBUG - encrypting file=/var/folders/59/ssms5v8s531flb9vk_0p9tgr0000gn/T/tmps30sb20g/file0.txt
2023-12-15 13:38:59,976 - ThreadPoolExecutor-2_0 encryption_util.py:138 - encrypt_file() - DEBUG - unencrypted file: /var/folders/59/ssms5v8s531flb9vk_0p9tgr0000gn/T/tmps30sb20g/file0.txt, temp file: /var/folders/59/ssms5v8s531flb9vk_0p9tgr0000gn/T/tmp3sql3d0b/file0.txt#864kb721, tmp_dir: /var/folders/59/ssms5v8s531flb9vk_0p9tgr0000gn/T/tmp3sql3d0b
2023-12-15 13:38:59,976 - ThreadPoolExecutor-2_0 encryption_util.py:75 - encrypt_stream() - DEBUG - key_size = 16
2023-12-15 13:38:59,976 - ThreadPoolExecutor-2_0 storage_client.py:234 - prepare_upload() - DEBUG - finished preprocessing
2023-12-15 13:38:59,976 - ThreadPoolExecutor-2_0 storage_client.py:239 - prepare_upload() - DEBUG - number of chunks 1
2023-12-15 13:38:59,976 - ThreadPoolExecutor-2_0 file_transfer_agent.py:482 - preprocess_done_cb() - DEBUG - Finished preparing file file0.txt
2023-12-15 13:38:59,977 - ThreadPoolExecutor-1_0 storage_client.py:413 - upload_chunk() - DEBUG - Uploading chunk 0 of file /var/folders/59/ssms5v8s531flb9vk_0p9tgr0000gn/T/tmp3sql3d0b/file0.txt#864kb721
2023-12-15 13:38:59,977 - ThreadPoolExecutor-1_0 network.py:1219 - _use_requests_session() - DEBUG - Session status for SessionPool 'kwiijjsfcb1stg.blob.core.windows.net', SessionPool 1/1 active sessions
2023-12-15 13:38:59,977 - ThreadPoolExecutor-1_0 storage_client.py:286 - _send_request_with_retry() - DEBUG - storage client request with session <snowflake.connector.vendored.requests.sessions.Session object at 0x155f57250>
2023-12-15 13:39:00,009 - ThreadPoolExecutor-1_0 connectionpool.py:474 - _make_request() - DEBUG - https://kwiijjsfcb1stg.blob.core.windows.net:443 "PUT /stageszza4b74fa3-7879-4751-aba8-ab66e1a896a6/file0.txt?sv=2021-04-10&spr=https&se=2023-12-15T14%3A38%3A59Z&sr=c&sp=rwdl&sig=GqwRGEr0C4QrTy2MSVbbbuq97naxVLd6aP%2BBgvoDiAk%3D HTTP/1.1" 201 0
2023-12-15 13:39:00,009 - ThreadPoolExecutor-1_0 network.py:1224 - _use_requests_session() - DEBUG - Session status for SessionPool 'kwiijjsfcb1stg.blob.core.windows.net', SessionPool 0/1 active sessions
2023-12-15 13:39:00,009 - ThreadPoolExecutor-1_0 storage_client.py:415 - upload_chunk() - DEBUG - Successfully uploaded chunk 0 of file /var/folders/59/ssms5v8s531flb9vk_0p9tgr0000gn/T/tmp3sql3d0b/file0.txt#864kb721
2023-12-15 13:39:00,009 - ThreadPoolExecutor-1_0 file_transfer_agent.py:528 - transfer_done_cb() - DEBUG - Chunk 0/1 of file file0.txt reached callback
2023-12-15 13:39:00,009 - ThreadPoolExecutor-1_0 file_transfer_agent.py:544 - transfer_done_cb() - DEBUG - Chunk progress: file0.txt: completed: 1 failed: 0 total: 1
2023-12-15 13:39:00,009 - ThreadPoolExecutor-1_0 storage_client.py:255 - finish_upload() - DEBUG - /var/folders/59/ssms5v8s531flb9vk_0p9tgr0000gn/T/tmps30sb20g/file0.txt upload is completed.
2023-12-15 13:39:00,009 - ThreadPoolExecutor-1_0 storage_client.py:449 - delete_client_data() - DEBUG - cleaning up tmp dir: /var/folders/59/ssms5v8s531flb9vk_0p9tgr0000gn/T/tmp3sql3d0b
2023-12-15 13:39:00,009 - MainThread file_transfer_agent.py:734 - result() - DEBUG - raise_put_get_error: True, UPLOADED, <class 'str'>, ResultStatus.ERROR, <enum 'ResultStatus'>
2023-12-15 13:39:00,009 - MainThread cursor.py:1191 - _init_result_and_meta() - DEBUG - Query result format: json
2023-12-15 13:39:00,009 - MainThread result_batch.py:487 - _parse() - DEBUG - parsing for result batch id: 1
2023-12-15 13:39:00,010 - MainThread pandas_tools.py:131 - _do_create_temp_file_format() - DEBUG - creating file format with 'CREATE TEMP FILE FORMAT "RAW"."DWD"."kmrtnsubmb" /* Python:snowflake.connector.pandas_tools.write_pandas() */ TYPE=PARQUET COMPRESSION=auto'
2023-12-15 13:39:00,010 - MainThread cursor.py:960 - execute() - DEBUG - executing SQL/command
2023-12-15 13:39:00,010 - MainThread cursor.py:816 - _preprocess_pyformat_query() - DEBUG - binding: [CREATE TEMP FILE FORMAT "RAW"."DWD"."kmrtnsubmb" /* Python:snowflake.connector.p...] with input=[None], processed=[{}]
2023-12-15 13:39:00,010 - MainThread cursor.py:1028 - execute() - INFO - query: [CREATE TEMP FILE FORMAT "RAW"."DWD"."kmrtnsubmb" /* Python:snowflake.connector.p...]
2023-12-15 13:39:00,010 - MainThread connection.py:1542 - _next_sequence_counter() - DEBUG - sequence counter: 3
2023-12-15 13:39:00,010 - MainThread cursor.py:690 - _execute_helper() - DEBUG - Request id: aed0f5b7-bbca-49aa-b417-cf255d0f7bab
2023-12-15 13:39:00,010 - MainThread cursor.py:692 - _execute_helper() - DEBUG - running query [CREATE TEMP FILE FORMAT "RAW"."DWD"."kmrtnsubmb" /* Python:snowflake.connector.p...]
2023-12-15 13:39:00,010 - MainThread cursor.py:699 - _execute_helper() - DEBUG - is_file_transfer: True
2023-12-15 13:39:00,010 - MainThread connection.py:1205 - cmd_query() - DEBUG - _cmd_query
2023-12-15 13:39:00,010 - MainThread _query_context_cache.py:155 - serialize_to_dict() - DEBUG - serialize_to_dict() called
2023-12-15 13:39:00,010 - MainThread _query_context_cache.py:274 - log_cache_entries() - DEBUG - Cache Entry: (0, 163192743929263, 0)
2023-12-15 13:39:00,010 - MainThread _query_context_cache.py:178 - serialize_to_dict() - DEBUG - serialize_to_dict(): data to send to server {'entries': [{'id': 0, 'timestamp': 163192743929263, 'priority': 0, 'context': {}}]}
2023-12-15 13:39:00,010 - MainThread connection.py:1232 - cmd_query() - DEBUG - sql=[CREATE TEMP FILE FORMAT "RAW"."DWD"."kmrtnsubmb" /* Python:snowflake.connector.p...], sequence_id=[3], is_file_transfer=[False]
2023-12-15 13:39:00,010 - MainThread network.py:1219 - _use_requests_session() - DEBUG - Session status for SessionPool '**censored**.snowflakecomputing.com', SessionPool 1/1 active sessions
2023-12-15 13:39:00,010 - MainThread network.py:870 - _request_exec_wrapper() - DEBUG - remaining request timeout: N/A ms, retry cnt: 1
2023-12-15 13:39:00,010 - MainThread network.py:852 - add_request_guid() - DEBUG - Request guid: 702183dc-d0bb-4a12-b299-de34183a77aa
2023-12-15 13:39:00,010 - MainThread network.py:1060 - _request_exec() - DEBUG - socket timeout: 60
2023-12-15 13:39:00,104 - MainThread connectionpool.py:474 - _make_request() - DEBUG - https://**censored**.snowflakecomputing.com:443 "POST /queries/v1/query-request?requestId=aed0f5b7-bbca-49aa-b417-cf255d0f7bab&request_guid=702183dc-d0bb-4a12-b299-de34183a77aa HTTP/1.1" 200 None
2023-12-15 13:39:00,104 - MainThread network.py:1087 - _request_exec() - DEBUG - SUCCESS
2023-12-15 13:39:00,104 - MainThread network.py:1224 - _use_requests_session() - DEBUG - Session status for SessionPool '**censored**.snowflakecomputing.com', SessionPool 0/1 active sessions
2023-12-15 13:39:00,104 - MainThread network.py:740 - _post_request() - DEBUG - ret[code] = None, after post request
2023-12-15 13:39:00,104 - MainThread network.py:766 - _post_request() - DEBUG - Query id: 01b10137-0203-38a4-0001-ab7a00304156
2023-12-15 13:39:00,104 - MainThread _query_context_cache.py:189 - deserialize_json_dict() - DEBUG - deserialize_json_dict() called: data from server: {'entries': [{'id': 0, 'timestamp': 163192744437936, 'priority': 0}]}
2023-12-15 13:39:00,104 - MainThread _query_context_cache.py:274 - log_cache_entries() - DEBUG - Cache Entry: (0, 163192743929263, 0)
2023-12-15 13:39:00,104 - MainThread _query_context_cache.py:230 - deserialize_json_dict() - DEBUG - deserialize {'id': 0, 'timestamp': 163192744437936, 'priority': 0}
2023-12-15 13:39:00,105 - MainThread _query_context_cache.py:101 - _sync_priority_map() - DEBUG - sync_priority_map called priority_map size = 0, new_priority_map size = 1
2023-12-15 13:39:00,105 - MainThread _query_context_cache.py:127 - trim_cache() - DEBUG - trim_cache() called. treeSet size is 1 and cache capacity is 5
2023-12-15 13:39:00,105 - MainThread _query_context_cache.py:136 - trim_cache() - DEBUG - trim_cache() returns. treeSet size is 1 and cache capacity is 5
2023-12-15 13:39:00,105 - MainThread _query_context_cache.py:269 - deserialize_json_dict() - DEBUG - deserialize_json_dict() returns
2023-12-15 13:39:00,105 - MainThread _query_context_cache.py:274 - log_cache_entries() - DEBUG - Cache Entry: (0, 163192744437936, 0)
2023-12-15 13:39:00,105 - MainThread cursor.py:1035 - execute() - DEBUG - sfqid: 01b10137-0203-38a4-0001-ab7a00304156
2023-12-15 13:39:00,105 - MainThread cursor.py:1041 - execute() - INFO - query execution done
2023-12-15 13:39:00,105 - MainThread cursor.py:1055 - execute() - DEBUG - SUCCESS
2023-12-15 13:39:00,105 - MainThread cursor.py:1074 - execute() - DEBUG - PUT OR GET: False
2023-12-15 13:39:00,105 - MainThread cursor.py:1191 - _init_result_and_meta() - DEBUG - Query result format: json
2023-12-15 13:39:00,105 - MainThread result_batch.py:487 - _parse() - DEBUG - parsing for result batch id: 1
2023-12-15 13:39:00,105 - MainThread cursor.py:1205 - _init_result_and_meta() - INFO - Number of results in first chunk: 1
2023-12-15 13:39:00,105 - MainThread pandas_tools.py:375 - write_pandas() - DEBUG - inferring schema with 'SELECT COLUMN_NAME, TYPE FROM table(infer_schema(location=>'@"RAW"."DWD"."cnoxjgjzco"', file_format=>'"RAW"."DWD"."kmrtnsubmb"'))'
2023-12-15 13:39:00,105 - MainThread cursor.py:960 - execute() - DEBUG - executing SQL/command
2023-12-15 13:39:00,105 - MainThread cursor.py:816 - _preprocess_pyformat_query() - DEBUG - binding: [SELECT COLUMN_NAME, TYPE FROM table(infer_schema(location=>'@"RAW"."DWD"."cnoxjg...] with input=[None], processed=[{}]
2023-12-15 13:39:00,105 - MainThread cursor.py:1028 - execute() - INFO - query: [SELECT COLUMN_NAME, TYPE FROM table(infer_schema(location=>'@"RAW"."DWD"."cnoxjg...]
2023-12-15 13:39:00,105 - MainThread connection.py:1542 - _next_sequence_counter() - DEBUG - sequence counter: 4
2023-12-15 13:39:00,105 - MainThread cursor.py:690 - _execute_helper() - DEBUG - Request id: 1d1008a5-7332-4854-bdb5-4b78eed30fc3
2023-12-15 13:39:00,105 - MainThread cursor.py:692 - _execute_helper() - DEBUG - running query [SELECT COLUMN_NAME, TYPE FROM table(infer_schema(location=>'@"RAW"."DWD"."cnoxjg...]
2023-12-15 13:39:00,105 - MainThread cursor.py:699 - _execute_helper() - DEBUG - is_file_transfer: True
2023-12-15 13:39:00,105 - MainThread connection.py:1205 - cmd_query() - DEBUG - _cmd_query
2023-12-15 13:39:00,105 - MainThread _query_context_cache.py:155 - serialize_to_dict() - DEBUG - serialize_to_dict() called
2023-12-15 13:39:00,105 - MainThread _query_context_cache.py:274 - log_cache_entries() - DEBUG - Cache Entry: (0, 163192744437936, 0)
2023-12-15 13:39:00,105 - MainThread _query_context_cache.py:178 - serialize_to_dict() - DEBUG - serialize_to_dict(): data to send to server {'entries': [{'id': 0, 'timestamp': 163192744437936, 'priority': 0, 'context': {}}]}
2023-12-15 13:39:00,105 - MainThread connection.py:1232 - cmd_query() - DEBUG - sql=[SELECT COLUMN_NAME, TYPE FROM table(infer_schema(location=>'@"RAW"."DWD"."cnoxjg...], sequence_id=[4], is_file_transfer=[False]
2023-12-15 13:39:00,105 - MainThread network.py:1219 - _use_requests_session() - DEBUG - Session status for SessionPool '**censored**.snowflakecomputing.com', SessionPool 1/1 active sessions
2023-12-15 13:39:00,105 - MainThread network.py:870 - _request_exec_wrapper() - DEBUG - remaining request timeout: N/A ms, retry cnt: 1
2023-12-15 13:39:00,105 - MainThread network.py:852 - add_request_guid() - DEBUG - Request guid: 9319ce58-990a-4190-ac06-166e02cefbe9
2023-12-15 13:39:00,105 - MainThread network.py:1060 - _request_exec() - DEBUG - socket timeout: 60
2023-12-15 13:39:00,507 - MainThread connectionpool.py:474 - _make_request() - DEBUG - https://**censored**.snowflakecomputing.com:443 "POST /queries/v1/query-request?requestId=1d1008a5-7332-4854-bdb5-4b78eed30fc3&request_guid=9319ce58-990a-4190-ac06-166e02cefbe9 HTTP/1.1" 200 None
2023-12-15 13:39:00,508 - MainThread network.py:1087 - _request_exec() - DEBUG - SUCCESS
2023-12-15 13:39:00,508 - MainThread network.py:1224 - _use_requests_session() - DEBUG - Session status for SessionPool '**censored**.snowflakecomputing.com', SessionPool 0/1 active sessions
2023-12-15 13:39:00,508 - MainThread network.py:740 - _post_request() - DEBUG - ret[code] = None, after post request
2023-12-15 13:39:00,508 - MainThread network.py:766 - _post_request() - DEBUG - Query id: 01b10137-0203-3874-0001-ab7a0030212a
2023-12-15 13:39:00,508 - MainThread _query_context_cache.py:189 - deserialize_json_dict() - DEBUG - deserialize_json_dict() called: data from server: {'entries': [{'id': 0, 'timestamp': 163192744826207, 'priority': 0}]}
2023-12-15 13:39:00,508 - MainThread _query_context_cache.py:274 - log_cache_entries() - DEBUG - Cache Entry: (0, 163192744437936, 0)
2023-12-15 13:39:00,508 - MainThread _query_context_cache.py:230 - deserialize_json_dict() - DEBUG - deserialize {'id': 0, 'timestamp': 163192744826207, 'priority': 0}
2023-12-15 13:39:00,508 - MainThread _query_context_cache.py:101 - _sync_priority_map() - DEBUG - sync_priority_map called priority_map size = 0, new_priority_map size = 1
2023-12-15 13:39:00,508 - MainThread _query_context_cache.py:127 - trim_cache() - DEBUG - trim_cache() called. treeSet size is 1 and cache capacity is 5
2023-12-15 13:39:00,508 - MainThread _query_context_cache.py:136 - trim_cache() - DEBUG - trim_cache() returns. treeSet size is 1 and cache capacity is 5
2023-12-15 13:39:00,508 - MainThread _query_context_cache.py:269 - deserialize_json_dict() - DEBUG - deserialize_json_dict() returns
2023-12-15 13:39:00,508 - MainThread _query_context_cache.py:274 - log_cache_entries() - DEBUG - Cache Entry: (0, 163192744826207, 0)
2023-12-15 13:39:00,508 - MainThread cursor.py:1035 - execute() - DEBUG - sfqid: 01b10137-0203-3874-0001-ab7a0030212a
2023-12-15 13:39:00,508 - MainThread cursor.py:1041 - execute() - INFO - query execution done
2023-12-15 13:39:00,508 - MainThread cursor.py:1055 - execute() - DEBUG - SUCCESS
2023-12-15 13:39:00,508 - MainThread cursor.py:1074 - execute() - DEBUG - PUT OR GET: False
2023-12-15 13:39:00,509 - MainThread cursor.py:1191 - _init_result_and_meta() - DEBUG - Query result format: arrow
2023-12-15 13:39:00,509 - MainThread cursor.py:1205 - _init_result_and_meta() - INFO - Number of results in first chunk: 2
2023-12-15 13:39:00,509 - MainThread result_batch.py:68 - _create_nanoarrow_iterator() - DEBUG - Using nanoarrow as the arrow data converter
2023-12-15 13:39:00,509 - MainThread CArrowIterator.cpp:53 - CArrowIterator() - DEBUG - Arrow BatchSize: 1
2023-12-15 13:39:00,509 - MainThread CArrowChunkIterator.cpp:39 - CArrowChunkIterator() - DEBUG - Arrow chunk info: batchCount 1, columnCount 2, use_numpy: 0
2023-12-15 13:39:00,509 - MainThread nanoarrow_arrow_iterator.cpython-310-darwin.so:0 - __cinit__() - DEBUG - Batches read: 0
2023-12-15 13:39:00,509 - MainThread result_set.py:59 - result_set_iterator() - DEBUG - beginning to schedule result batch downloads
2023-12-15 13:39:00,509 - MainThread CArrowChunkIterator.cpp:66 - next() - DEBUG - Current batch index: 0, rows in current batch: 2
2023-12-15 13:39:00,509 - MainThread pandas_tools.py:401 - write_pandas() - DEBUG - auto creating table with 'CREATE TRANSIENT TABLE IF NOT EXISTS "RAW"."DWD"."uchkqoikdh" ("col1" NUMBER(38, 0), "col2" TEXT) /* Python:snowflake.connector.pandas_tools.write_pandas() */ '
2023-12-15 13:39:00,509 - MainThread cursor.py:960 - execute() - DEBUG - executing SQL/command
2023-12-15 13:39:00,509 - MainThread cursor.py:816 - _preprocess_pyformat_query() - DEBUG - binding: [CREATE TRANSIENT TABLE IF NOT EXISTS "RAW"."DWD"."uchkqoikdh" ("col1" NUMBER(38,...] with input=[None], processed=[{}]
2023-12-15 13:39:00,509 - MainThread cursor.py:1028 - execute() - INFO - query: [CREATE TRANSIENT TABLE IF NOT EXISTS "RAW"."DWD"."uchkqoikdh" ("col1" NUMBER(38,...]
2023-12-15 13:39:00,509 - MainThread connection.py:1542 - _next_sequence_counter() - DEBUG - sequence counter: 5
2023-12-15 13:39:00,509 - MainThread cursor.py:690 - _execute_helper() - DEBUG - Request id: 866e60f5-5b5e-4c46-99e7-b5b55f0427ec
2023-12-15 13:39:00,510 - MainThread cursor.py:692 - _execute_helper() - DEBUG - running query [CREATE TRANSIENT TABLE IF NOT EXISTS "RAW"."DWD"."uchkqoikdh" ("col1" NUMBER(38,...]
2023-12-15 13:39:00,510 - MainThread cursor.py:699 - _execute_helper() - DEBUG - is_file_transfer: True
2023-12-15 13:39:00,510 - MainThread connection.py:1205 - cmd_query() - DEBUG - _cmd_query
2023-12-15 13:39:00,510 - MainThread _query_context_cache.py:155 - serialize_to_dict() - DEBUG - serialize_to_dict() called
2023-12-15 13:39:00,510 - MainThread _query_context_cache.py:274 - log_cache_entries() - DEBUG - Cache Entry: (0, 163192744826207, 0)
2023-12-15 13:39:00,510 - MainThread _query_context_cache.py:178 - serialize_to_dict() - DEBUG - serialize_to_dict(): data to send to server {'entries': [{'id': 0, 'timestamp': 163192744826207, 'priority': 0, 'context': {}}]}
2023-12-15 13:39:00,510 - MainThread connection.py:1232 - cmd_query() - DEBUG - sql=[CREATE TRANSIENT TABLE IF NOT EXISTS "RAW"."DWD"."uchkqoikdh" ("col1" NUMBER(38,...], sequence_id=[5], is_file_transfer=[False]
2023-12-15 13:39:00,510 - MainThread network.py:1219 - _use_requests_session() - DEBUG - Session status for SessionPool '**censored**.snowflakecomputing.com', SessionPool 1/1 active sessions
2023-12-15 13:39:00,510 - MainThread network.py:870 - _request_exec_wrapper() - DEBUG - remaining request timeout: N/A ms, retry cnt: 1
2023-12-15 13:39:00,510 - MainThread network.py:852 - add_request_guid() - DEBUG - Request guid: af173138-81ba-4293-a041-1c0029c8aa3d
2023-12-15 13:39:00,510 - MainThread network.py:1060 - _request_exec() - DEBUG - socket timeout: 60
2023-12-15 13:39:00,826 - MainThread connectionpool.py:474 - _make_request() - DEBUG - https://**censored**.snowflakecomputing.com:443 "POST /queries/v1/query-request?requestId=866e60f5-5b5e-4c46-99e7-b5b55f0427ec&request_guid=af173138-81ba-4293-a041-1c0029c8aa3d HTTP/1.1" 200 None
2023-12-15 13:39:00,827 - MainThread network.py:1087 - _request_exec() - DEBUG - SUCCESS
2023-12-15 13:39:00,827 - MainThread network.py:1224 - _use_requests_session() - DEBUG - Session status for SessionPool '**censored**.snowflakecomputing.com', SessionPool 0/1 active sessions
2023-12-15 13:39:00,827 - MainThread network.py:740 - _post_request() - DEBUG - ret[code] = None, after post request
2023-12-15 13:39:00,827 - MainThread network.py:766 - _post_request() - DEBUG - Query id: 01b10137-0203-3874-0001-ab7a0030212e
2023-12-15 13:39:00,827 - MainThread _query_context_cache.py:189 - deserialize_json_dict() - DEBUG - deserialize_json_dict() called: data from server: {'entries': [{'id': 0, 'timestamp': 163192745152932, 'priority': 0}]}
2023-12-15 13:39:00,827 - MainThread _query_context_cache.py:274 - log_cache_entries() - DEBUG - Cache Entry: (0, 163192744826207, 0)
2023-12-15 13:39:00,827 - MainThread _query_context_cache.py:230 - deserialize_json_dict() - DEBUG - deserialize {'id': 0, 'timestamp': 163192745152932, 'priority': 0}
2023-12-15 13:39:00,827 - MainThread _query_context_cache.py:101 - _sync_priority_map() - DEBUG - sync_priority_map called priority_map size = 0, new_priority_map size = 1
2023-12-15 13:39:00,827 - MainThread _query_context_cache.py:127 - trim_cache() - DEBUG - trim_cache() called. treeSet size is 1 and cache capacity is 5
2023-12-15 13:39:00,827 - MainThread _query_context_cache.py:136 - trim_cache() - DEBUG - trim_cache() returns. treeSet size is 1 and cache capacity is 5
2023-12-15 13:39:00,827 - MainThread _query_context_cache.py:269 - deserialize_json_dict() - DEBUG - deserialize_json_dict() returns
2023-12-15 13:39:00,827 - MainThread _query_context_cache.py:274 - log_cache_entries() - DEBUG - Cache Entry: (0, 163192745152932, 0)
2023-12-15 13:39:00,827 - MainThread cursor.py:1035 - execute() - DEBUG - sfqid: 01b10137-0203-3874-0001-ab7a0030212e
2023-12-15 13:39:00,828 - MainThread cursor.py:1041 - execute() - INFO - query execution done
2023-12-15 13:39:00,828 - MainThread cursor.py:1055 - execute() - DEBUG - SUCCESS
2023-12-15 13:39:00,828 - MainThread cursor.py:1074 - execute() - DEBUG - PUT OR GET: False
2023-12-15 13:39:00,828 - MainThread cursor.py:1191 - _init_result_and_meta() - DEBUG - Query result format: json
2023-12-15 13:39:00,828 - MainThread result_batch.py:487 - _parse() - DEBUG - parsing for result batch id: 1
2023-12-15 13:39:00,828 - MainThread cursor.py:1205 - _init_result_and_meta() - INFO - Number of results in first chunk: 1
2023-12-15 13:39:00,828 - MainThread pandas_tools.py:432 - write_pandas() - DEBUG - copying into with 'COPY INTO "RAW"."DWD"."uchkqoikdh" /* Python:snowflake.connector.pandas_tools.write_pandas() */ ("col1","col2") FROM (SELECT $1:"col1"::NUMBER(38, 0),$1:"col2"::TEXT FROM @"RAW"."DWD"."cnoxjgjzco") FILE_FORMAT=(TYPE=PARQUET COMPRESSION=auto BINARY_AS_TEXT=FALSE) PURGE=TRUE ON_ERROR=abort_statement'
2023-12-15 13:39:00,828 - MainThread cursor.py:960 - execute() - DEBUG - executing SQL/command
2023-12-15 13:39:00,828 - MainThread cursor.py:816 - _preprocess_pyformat_query() - DEBUG - binding: [COPY INTO "RAW"."DWD"."uchkqoikdh" /* Python:snowflake.connector.pandas_tools.wr...] with input=[None], processed=[{}]
2023-12-15 13:39:00,828 - MainThread cursor.py:1028 - execute() - INFO - query: [COPY INTO "RAW"."DWD"."uchkqoikdh" /* Python:snowflake.connector.pandas_tools.wr...]
2023-12-15 13:39:00,828 - MainThread connection.py:1542 - _next_sequence_counter() - DEBUG - sequence counter: 6
2023-12-15 13:39:00,828 - MainThread cursor.py:690 - _execute_helper() - DEBUG - Request id: f4a212b4-bb2d-46fd-b018-e944dbfa312f
2023-12-15 13:39:00,828 - MainThread cursor.py:692 - _execute_helper() - DEBUG - running query [COPY INTO "RAW"."DWD"."uchkqoikdh" /* Python:snowflake.connector.pandas_tools.wr...]
2023-12-15 13:39:00,828 - MainThread cursor.py:699 - _execute_helper() - DEBUG - is_file_transfer: True
2023-12-15 13:39:00,828 - MainThread connection.py:1205 - cmd_query() - DEBUG - _cmd_query
2023-12-15 13:39:00,828 - MainThread _query_context_cache.py:155 - serialize_to_dict() - DEBUG - serialize_to_dict() called
2023-12-15 13:39:00,828 - MainThread _query_context_cache.py:274 - log_cache_entries() - DEBUG - Cache Entry: (0, 163192745152932, 0)
2023-12-15 13:39:00,828 - MainThread _query_context_cache.py:178 - serialize_to_dict() - DEBUG - serialize_to_dict(): data to send to server {'entries': [{'id': 0, 'timestamp': 163192745152932, 'priority': 0, 'context': {}}]}
2023-12-15 13:39:00,828 - MainThread connection.py:1232 - cmd_query() - DEBUG - sql=[COPY INTO "RAW"."DWD"."uchkqoikdh" /* Python:snowflake.connector.pandas_tools.wr...], sequence_id=[6], is_file_transfer=[False]
2023-12-15 13:39:00,828 - MainThread network.py:1219 - _use_requests_session() - DEBUG - Session status for SessionPool '**censored**.snowflakecomputing.com', SessionPool 1/1 active sessions
2023-12-15 13:39:00,828 - MainThread network.py:870 - _request_exec_wrapper() - DEBUG - remaining request timeout: N/A ms, retry cnt: 1
2023-12-15 13:39:00,828 - MainThread network.py:852 - add_request_guid() - DEBUG - Request guid: e30fe086-8f00-4aac-afca-6ab9dec2fc75
2023-12-15 13:39:00,828 - MainThread network.py:1060 - _request_exec() - DEBUG - socket timeout: 60
2023-12-15 13:39:01,651 - MainThread connectionpool.py:474 - _make_request() - DEBUG - https://**censored**.snowflakecomputing.com:443 "POST /queries/v1/query-request?requestId=f4a212b4-bb2d-46fd-b018-e944dbfa312f&request_guid=e30fe086-8f00-4aac-afca-6ab9dec2fc75 HTTP/1.1" 200 None
2023-12-15 13:39:01,653 - MainThread network.py:1087 - _request_exec() - DEBUG - SUCCESS
2023-12-15 13:39:01,653 - MainThread network.py:1224 - _use_requests_session() - DEBUG - Session status for SessionPool '**censored**.snowflakecomputing.com', SessionPool 0/1 active sessions
2023-12-15 13:39:01,653 - MainThread network.py:740 - _post_request() - DEBUG - ret[code] = None, after post request
2023-12-15 13:39:01,653 - MainThread network.py:766 - _post_request() - DEBUG - Query id: 01b10137-0203-3874-0001-ab7a00302132
2023-12-15 13:39:01,654 - MainThread _query_context_cache.py:189 - deserialize_json_dict() - DEBUG - deserialize_json_dict() called: data from server: {'entries': [{'id': 0, 'timestamp': 163192745971038, 'priority': 0}]}
2023-12-15 13:39:01,654 - MainThread _query_context_cache.py:274 - log_cache_entries() - DEBUG - Cache Entry: (0, 163192745152932, 0)
2023-12-15 13:39:01,654 - MainThread _query_context_cache.py:230 - deserialize_json_dict() - DEBUG - deserialize {'id': 0, 'timestamp': 163192745971038, 'priority': 0}
2023-12-15 13:39:01,654 - MainThread _query_context_cache.py:101 - _sync_priority_map() - DEBUG - sync_priority_map called priority_map size = 0, new_priority_map size = 1
2023-12-15 13:39:01,654 - MainThread _query_context_cache.py:127 - trim_cache() - DEBUG - trim_cache() called. treeSet size is 1 and cache capacity is 5
2023-12-15 13:39:01,654 - MainThread _query_context_cache.py:136 - trim_cache() - DEBUG - trim_cache() returns. treeSet size is 1 and cache capacity is 5
2023-12-15 13:39:01,654 - MainThread _query_context_cache.py:269 - deserialize_json_dict() - DEBUG - deserialize_json_dict() returns
2023-12-15 13:39:01,654 - MainThread _query_context_cache.py:274 - log_cache_entries() - DEBUG - Cache Entry: (0, 163192745971038, 0)
2023-12-15 13:39:01,654 - MainThread cursor.py:1035 - execute() - DEBUG - sfqid: 01b10137-0203-3874-0001-ab7a00302132
2023-12-15 13:39:01,654 - MainThread cursor.py:1041 - execute() - INFO - query execution done
2023-12-15 13:39:01,654 - MainThread cursor.py:1055 - execute() - DEBUG - SUCCESS
2023-12-15 13:39:01,654 - MainThread cursor.py:1074 - execute() - DEBUG - PUT OR GET: False
2023-12-15 13:39:01,654 - MainThread cursor.py:1191 - _init_result_and_meta() - DEBUG - Query result format: json
2023-12-15 13:39:01,654 - MainThread result_batch.py:487 - _parse() - DEBUG - parsing for result batch id: 1
2023-12-15 13:39:01,654 - MainThread cursor.py:1205 - _init_result_and_meta() - INFO - Number of results in first chunk: 1
2023-12-15 13:39:01,655 - MainThread result_set.py:59 - result_set_iterator() - DEBUG - beginning to schedule result batch downloads
2023-12-15 13:39:01,655 - MainThread pandas_tools.py:362 - drop_object() - DEBUG - dropping table with 'DROP TABLE IF EXISTS "RAW"."DWD"."TEST" /* Python:snowflake.connector.pandas_tools.write_pandas() */'
2023-12-15 13:39:01,655 - MainThread cursor.py:960 - execute() - DEBUG - executing SQL/command
2023-12-15 13:39:01,655 - MainThread cursor.py:816 - _preprocess_pyformat_query() - DEBUG - binding: [DROP TABLE IF EXISTS "RAW"."DWD"."TEST" /* Python:snowflake.connector.pandas_too...] with input=[None], processed=[{}]
2023-12-15 13:39:01,655 - MainThread cursor.py:1028 - execute() - INFO - query: [DROP TABLE IF EXISTS "RAW"."DWD"."TEST" /* Python:snowflake.connector.pandas_too...]
2023-12-15 13:39:01,655 - MainThread connection.py:1542 - _next_sequence_counter() - DEBUG - sequence counter: 7
2023-12-15 13:39:01,655 - MainThread cursor.py:690 - _execute_helper() - DEBUG - Request id: 7b9b7e90-88c3-4a7f-886f-f33e7757321e
2023-12-15 13:39:01,655 - MainThread cursor.py:692 - _execute_helper() - DEBUG - running query [DROP TABLE IF EXISTS "RAW"."DWD"."TEST" /* Python:snowflake.connector.pandas_too...]
2023-12-15 13:39:01,655 - MainThread cursor.py:699 - _execute_helper() - DEBUG - is_file_transfer: True
2023-12-15 13:39:01,655 - MainThread connection.py:1205 - cmd_query() - DEBUG - _cmd_query
2023-12-15 13:39:01,655 - MainThread _query_context_cache.py:155 - serialize_to_dict() - DEBUG - serialize_to_dict() called
2023-12-15 13:39:01,655 - MainThread _query_context_cache.py:274 - log_cache_entries() - DEBUG - Cache Entry: (0, 163192745971038, 0)
2023-12-15 13:39:01,655 - MainThread _query_context_cache.py:178 - serialize_to_dict() - DEBUG - serialize_to_dict(): data to send to server {'entries': [{'id': 0, 'timestamp': 163192745971038, 'priority': 0, 'context': {}}]}
2023-12-15 13:39:01,655 - MainThread connection.py:1232 - cmd_query() - DEBUG - sql=[DROP TABLE IF EXISTS "RAW"."DWD"."TEST" /* Python:snowflake.connector.pandas_too...], sequence_id=[7], is_file_transfer=[False]
2023-12-15 13:39:01,655 - MainThread network.py:1219 - _use_requests_session() - DEBUG - Session status for SessionPool '**censored**.snowflakecomputing.com', SessionPool 1/1 active sessions
2023-12-15 13:39:01,656 - MainThread network.py:870 - _request_exec_wrapper() - DEBUG - remaining request timeout: N/A ms, retry cnt: 1
2023-12-15 13:39:01,656 - MainThread network.py:852 - add_request_guid() - DEBUG - Request guid: 8e52c0d2-4a67-4650-9c55-a171774c7ad7
2023-12-15 13:39:01,656 - MainThread network.py:1060 - _request_exec() - DEBUG - socket timeout: 60
2023-12-15 13:39:01,708 - MainThread connectionpool.py:474 - _make_request() - DEBUG - https://**censored**.snowflakecomputing.com:443 "POST /queries/v1/query-request?requestId=7b9b7e90-88c3-4a7f-886f-f33e7757321e&request_guid=8e52c0d2-4a67-4650-9c55-a171774c7ad7 HTTP/1.1" 200 None
2023-12-15 13:39:01,708 - MainThread network.py:1087 - _request_exec() - DEBUG - SUCCESS
2023-12-15 13:39:01,708 - MainThread network.py:1224 - _use_requests_session() - DEBUG - Session status for SessionPool '**censored**.snowflakecomputing.com', SessionPool 0/1 active sessions
2023-12-15 13:39:01,709 - MainThread network.py:740 - _post_request() - DEBUG - ret[code] = None, after post request
2023-12-15 13:39:01,709 - MainThread network.py:766 - _post_request() - DEBUG - Query id: 01b10137-0203-3874-0001-ab7a00302136
2023-12-15 13:39:01,709 - MainThread _query_context_cache.py:189 - deserialize_json_dict() - DEBUG - deserialize_json_dict() called: data from server: {'entries': [{'id': 0, 'timestamp': 163192746037835, 'priority': 0}]}
2023-12-15 13:39:01,709 - MainThread _query_context_cache.py:274 - log_cache_entries() - DEBUG - Cache Entry: (0, 163192745971038, 0)
2023-12-15 13:39:01,709 - MainThread _query_context_cache.py:230 - deserialize_json_dict() - DEBUG - deserialize {'id': 0, 'timestamp': 163192746037835, 'priority': 0}
2023-12-15 13:39:01,709 - MainThread _query_context_cache.py:101 - _sync_priority_map() - DEBUG - sync_priority_map called priority_map size = 0, new_priority_map size = 1
2023-12-15 13:39:01,709 - MainThread _query_context_cache.py:127 - trim_cache() - DEBUG - trim_cache() called. treeSet size is 1 and cache capacity is 5
2023-12-15 13:39:01,709 - MainThread _query_context_cache.py:136 - trim_cache() - DEBUG - trim_cache() returns. treeSet size is 1 and cache capacity is 5
2023-12-15 13:39:01,709 - MainThread _query_context_cache.py:269 - deserialize_json_dict() - DEBUG - deserialize_json_dict() returns
2023-12-15 13:39:01,709 - MainThread _query_context_cache.py:274 - log_cache_entries() - DEBUG - Cache Entry: (0, 163192746037835, 0)
2023-12-15 13:39:01,709 - MainThread cursor.py:1035 - execute() - DEBUG - sfqid: 01b10137-0203-3874-0001-ab7a00302136
2023-12-15 13:39:01,709 - MainThread cursor.py:1041 - execute() - INFO - query execution done
2023-12-15 13:39:01,709 - MainThread cursor.py:1055 - execute() - DEBUG - SUCCESS
2023-12-15 13:39:01,709 - MainThread cursor.py:1074 - execute() - DEBUG - PUT OR GET: False
2023-12-15 13:39:01,709 - MainThread cursor.py:1191 - _init_result_and_meta() - DEBUG - Query result format: json
2023-12-15 13:39:01,709 - MainThread result_batch.py:487 - _parse() - DEBUG - parsing for result batch id: 1
2023-12-15 13:39:01,709 - MainThread cursor.py:1205 - _init_result_and_meta() - INFO - Number of results in first chunk: 1
2023-12-15 13:39:01,709 - MainThread pandas_tools.py:444 - write_pandas() - DEBUG - rename table with 'ALTER TABLE "RAW"."DWD"."uchkqoikdh" RENAME TO "RAW"."DWD"."TEST" /* Python:snowflake.connector.pandas_tools.write_pandas() */'
2023-12-15 13:39:01,709 - MainThread cursor.py:960 - execute() - DEBUG - executing SQL/command
2023-12-15 13:39:01,709 - MainThread cursor.py:816 - _preprocess_pyformat_query() - DEBUG - binding: [ALTER TABLE "RAW"."DWD"."uchkqoikdh" RENAME TO "RAW"."DWD"."TEST" /* Python:snow...] with input=[None], processed=[{}]
2023-12-15 13:39:01,710 - MainThread cursor.py:1028 - execute() - INFO - query: [ALTER TABLE "RAW"."DWD"."uchkqoikdh" RENAME TO "RAW"."DWD"."TEST" /* Python:snow...]
2023-12-15 13:39:01,710 - MainThread connection.py:1542 - _next_sequence_counter() - DEBUG - sequence counter: 8
2023-12-15 13:39:01,710 - MainThread cursor.py:690 - _execute_helper() - DEBUG - Request id: 223676f1-d3b8-4060-9373-d067a23a8c3d
2023-12-15 13:39:01,710 - MainThread cursor.py:692 - _execute_helper() - DEBUG - running query [ALTER TABLE "RAW"."DWD"."uchkqoikdh" RENAME TO "RAW"."DWD"."TEST" /* Python:snow...]
2023-12-15 13:39:01,710 - MainThread cursor.py:699 - _execute_helper() - DEBUG - is_file_transfer: True
2023-12-15 13:39:01,710 - MainThread connection.py:1205 - cmd_query() - DEBUG - _cmd_query
2023-12-15 13:39:01,710 - MainThread _query_context_cache.py:155 - serialize_to_dict() - DEBUG - serialize_to_dict() called
2023-12-15 13:39:01,710 - MainThread _query_context_cache.py:274 - log_cache_entries() - DEBUG - Cache Entry: (0, 163192746037835, 0)
2023-12-15 13:39:01,710 - MainThread _query_context_cache.py:178 - serialize_to_dict() - DEBUG - serialize_to_dict(): data to send to server {'entries': [{'id': 0, 'timestamp': 163192746037835, 'priority': 0, 'context': {}}]}
2023-12-15 13:39:01,710 - MainThread connection.py:1232 - cmd_query() - DEBUG - sql=[ALTER TABLE "RAW"."DWD"."uchkqoikdh" RENAME TO "RAW"."DWD"."TEST" /* Python:snow...], sequence_id=[8], is_file_transfer=[False]
2023-12-15 13:39:01,710 - MainThread network.py:1219 - _use_requests_session() - DEBUG - Session status for SessionPool '**censored**.snowflakecomputing.com', SessionPool 1/1 active sessions
2023-12-15 13:39:01,710 - MainThread network.py:870 - _request_exec_wrapper() - DEBUG - remaining request timeout: N/A ms, retry cnt: 1
2023-12-15 13:39:01,710 - MainThread network.py:852 - add_request_guid() - DEBUG - Request guid: d5c5e791-ccc1-4541-a7af-08d97f2a3745
2023-12-15 13:39:01,710 - MainThread network.py:1060 - _request_exec() - DEBUG - socket timeout: 60
2023-12-15 13:39:01,829 - MainThread connectionpool.py:474 - _make_request() - DEBUG - https://**censored**.snowflakecomputing.com:443 "POST /queries/v1/query-request?requestId=223676f1-d3b8-4060-9373-d067a23a8c3d&request_guid=d5c5e791-ccc1-4541-a7af-08d97f2a3745 HTTP/1.1" 200 None
2023-12-15 13:39:01,831 - MainThread network.py:1087 - _request_exec() - DEBUG - SUCCESS
2023-12-15 13:39:01,831 - MainThread network.py:1224 - _use_requests_session() - DEBUG - Session status for SessionPool '**censored**.snowflakecomputing.com', SessionPool 0/1 active sessions
2023-12-15 13:39:01,831 - MainThread network.py:740 - _post_request() - DEBUG - ret[code] = None, after post request
2023-12-15 13:39:01,831 - MainThread network.py:766 - _post_request() - DEBUG - Query id: 01b10137-0203-375f-0001-ab7a00300942
2023-12-15 13:39:01,831 - MainThread _query_context_cache.py:189 - deserialize_json_dict() - DEBUG - deserialize_json_dict() called: data from server: {'entries': [{'id': 0, 'timestamp': 163192746156524, 'priority': 0}]}
2023-12-15 13:39:01,831 - MainThread _query_context_cache.py:274 - log_cache_entries() - DEBUG - Cache Entry: (0, 163192746037835, 0)
2023-12-15 13:39:01,831 - MainThread _query_context_cache.py:230 - deserialize_json_dict() - DEBUG - deserialize {'id': 0, 'timestamp': 163192746156524, 'priority': 0}
2023-12-15 13:39:01,832 - MainThread _query_context_cache.py:101 - _sync_priority_map() - DEBUG - sync_priority_map called priority_map size = 0, new_priority_map size = 1
2023-12-15 13:39:01,832 - MainThread _query_context_cache.py:127 - trim_cache() - DEBUG - trim_cache() called. treeSet size is 1 and cache capacity is 5
2023-12-15 13:39:01,832 - MainThread _query_context_cache.py:136 - trim_cache() - DEBUG - trim_cache() returns. treeSet size is 1 and cache capacity is 5
2023-12-15 13:39:01,832 - MainThread _query_context_cache.py:269 - deserialize_json_dict() - DEBUG - deserialize_json_dict() returns
2023-12-15 13:39:01,832 - MainThread _query_context_cache.py:274 - log_cache_entries() - DEBUG - Cache Entry: (0, 163192746156524, 0)
2023-12-15 13:39:01,832 - MainThread cursor.py:1035 - execute() - DEBUG - sfqid: 01b10137-0203-375f-0001-ab7a00300942
2023-12-15 13:39:01,832 - MainThread cursor.py:1041 - execute() - INFO - query execution done
2023-12-15 13:39:01,832 - MainThread cursor.py:1055 - execute() - DEBUG - SUCCESS
2023-12-15 13:39:01,832 - MainThread cursor.py:1074 - execute() - DEBUG - PUT OR GET: False
2023-12-15 13:39:01,832 - MainThread cursor.py:1191 - _init_result_and_meta() - DEBUG - Query result format: json
2023-12-15 13:39:01,832 - MainThread result_batch.py:487 - _parse() - DEBUG - parsing for result batch id: 1
2023-12-15 13:39:01,832 - MainThread cursor.py:1205 - _init_result_and_meta() - INFO - Number of results in first chunk: 1
2023-12-15 13:39:01,832 - MainThread connection.py:718 - close() - INFO - closed
2023-12-15 13:39:01,832 - MainThread telemetry.py:211 - close() - DEBUG - Closing telemetry client.
2023-12-15 13:39:01,834 - MainThread telemetry.py:176 - send_batch() - DEBUG - Sending 1 logs to telemetry. Data is {'logs': [{'message': {'driver_type': 'PythonConnector', 'driver_version': '3.6.0', 'source': 'PythonConnector', 'type': 'client_imported_packages', 'value': "{'keyring', 'functools', 'sortedcontainers', 'ntpath', 'snowflake', 'typing_extensions', 'operator', 'asn1crypto', 'ast', 'socket', 'site', 'sre_parse', 'requests', 'cryptography', 'mmap', 'reprlib', 'grp', 'shutil', 'platformdirs', 'select', 'pyarrow', 'pandas', 'posix', 'mimetypes', 'importlib', 'selectors', 'gc', 'numpy', 'types', 'pathlib', 'cmath', 'heapq', 'queue', 'contextlib', 'dateutil', 'signal', 'pprint', 'unicodedata', 'xml', 'token', 'sys', 'zoneinfo', 'tokenize', 'string', 'math', 'urllib', 'fcntl', 'uuid', 'warnings', 'jwt', 'fractions', 'zipp', 'ipaddress', 'locale', 'base64', 'logging', 'http', 'pwd', 'subprocess', 'tomlkit', 'zlib', 'weakref', 'decimal', 'bisect', 'errno', 'binascii', 'stringprep', 'packaging', 'struct', 'importlib_metadata', 'zipfile', 'filelock', 'copy', 'linecache', 'tempfile', 'datetime', 'time', 'posixpath', 'stat', 'keyword', 'abc', 're', 'dataclasses', 'fnmatch', 'plistlib', 'quopri', 'sysconfig', 'inspect', 'threading', 'traceback', 'cython_runtime', 'uu', 'array', 'os', 'encodings', 'idna', 'copyreg', 'collections', 'lzma', 'sre_compile', 'pkgutil', 'atexit', 'contextvars', 'difflib', 'pydoc', 'hashlib', 'zipimport', 'textwrap', 'html', 'random', 'io', 'csv', 'configparser', 'concurrent', 'itertools', 'typing', 'charset_normalizer', 'marshal', 'six', 'codecs', 'certifi', 'shlex', 'secrets', 'enum', 'sre_constants', 'numbers', 'opcode', 'email', 'dis', 'pyexpat', 'builtins', 'platform', 'pytz', 'tarfile', 'gzip', 'json', 'webbrowser', 'jaraco', 'bz2', 'ctypes', 'urllib3', 'hmac', 'OpenSSL', 'ssl', 'genericpath', 'calendar', 'pickle'}"}, 'timestamp': '1702643939422'}, {'message': {'driver_type': 'PythonConnector', 'driver_version': '3.6.0', 'source': 'PythonConnector', 'type': 'client_time_consume_first_result', 'query_id': '01b10136-0203-38aa-0001-ab7a0030115a', 'value': -35}, 'timestamp': '1702643939595'}, {'message': {'driver_type': 'PythonConnector', 'driver_version': '3.6.0', 'source': 'PythonConnector', 'type': 'client_time_consume_last_result', 'query_id': '01b10136-0203-38aa-0001-ab7a0030115a', 'value': 0}, 'timestamp': '1702643939595'}, {'message': {'driver_type': 'PythonConnector', 'driver_version': '3.6.0', 'source': 'PythonConnector', 'type': 'client_time_consume_first_result', 'query_id': '01b10137-0203-38a4-0001-ab7a00304156', 'value': -35}, 'timestamp': '1702643940105'}, {'message': {'driver_type': 'PythonConnector', 'driver_version': '3.6.0', 'source': 'PythonConnector', 'type': 'client_time_consume_first_result', 'query_id': '01b10137-0203-3874-0001-ab7a0030212a', 'value': -31}, 'timestamp': '1702643940508'}, {'message': {'driver_type': 'PythonConnector', 'driver_version': '3.6.0', 'source': 'PythonConnector', 'type': 'client_time_consume_last_result', 'query_id': '01b10137-0203-3874-0001-ab7a0030212a', 'value': 1}, 'timestamp': '1702643940509'}, {'message': {'driver_type': 'PythonConnector', 'driver_version': '3.6.0', 'source': 'PythonConnector', 'type': 'client_time_consume_first_result', 'query_id': '01b10137-0203-3874-0001-ab7a0030212e', 'value': -34}, 'timestamp': '1702643940828'}, {'message': {'driver_type': 'PythonConnector', 'driver_version': '3.6.0', 'source': 'PythonConnector', 'type': 'client_time_consume_first_result', 'query_id': '01b10137-0203-3874-0001-ab7a00302132', 'value': -27}, 'timestamp': '1702643941654'}, {'message': {'driver_type': 'PythonConnector', 'driver_version': '3.6.0', 'source': 'PythonConnector', 'type': 'client_time_consume_last_result', 'query_id': '01b10137-0203-3874-0001-ab7a00302132', 'value': 1}, 'timestamp': '1702643941655'}, {'message': {'driver_type': 'PythonConnector', 'driver_version': '3.6.0', 'source': 'PythonConnector', 'type': 'client_time_consume_first_result', 'query_id': '01b10137-0203-3874-0001-ab7a00302136', 'value': -36}, 'timestamp': '1702643941709'}, {'message': {'driver_type': 'PythonConnector', 'driver_version': '3.6.0', 'source': 'PythonConnector', 'type': 'client_time_consume_first_result', 'query_id': '01b10137-0203-375f-0001-ab7a00300942', 'value': -33}, 'timestamp': '1702643941832'}, {'message': {'driver_type': 'PythonConnector', 'driver_version': '3.6.0', 'source': 'PythonConnector', 'type': 'client_write_pandas', 'query_id': '01b10137-0203-375f-0001-ab7a00300942', 'value': 1}, 'timestamp': '1702643941832'}]}.
2023-12-15 13:39:01,835 - MainThread network.py:1219 - _use_requests_session() - DEBUG - Session status for SessionPool '**censored**.snowflakecomputing.com', SessionPool 1/1 active sessions
2023-12-15 13:39:01,835 - MainThread network.py:870 - _request_exec_wrapper() - DEBUG - remaining request timeout: 5000 ms, retry cnt: 1
2023-12-15 13:39:01,835 - MainThread network.py:852 - add_request_guid() - DEBUG - Request guid: a4c3168c-e4a2-4d0f-b3b4-c69183c69e91
2023-12-15 13:39:01,835 - MainThread network.py:1060 - _request_exec() - DEBUG - socket timeout: 60
2023-12-15 13:39:01,871 - MainThread connectionpool.py:474 - _make_request() - DEBUG - https://**censored**.snowflakecomputing.com:443 "POST /telemetry/send?request_guid=a4c3168c-e4a2-4d0f-b3b4-c69183c69e91 HTTP/1.1" 200 None
2023-12-15 13:39:01,871 - MainThread network.py:1087 - _request_exec() - DEBUG - SUCCESS
2023-12-15 13:39:01,871 - MainThread network.py:1224 - _use_requests_session() - DEBUG - Session status for SessionPool '**censored**.snowflakecomputing.com', SessionPool 0/1 active sessions
2023-12-15 13:39:01,871 - MainThread network.py:740 - _post_request() - DEBUG - ret[code] = None, after post request
2023-12-15 13:39:01,871 - MainThread telemetry.py:200 - send_batch() - DEBUG - Successfully uploading metrics to telemetry.
2023-12-15 13:39:01,871 - MainThread connection.py:724 - close() - INFO - No async queries seem to be running, deleting session
2023-12-15 13:39:01,871 - MainThread network.py:1219 - _use_requests_session() - DEBUG - Session status for SessionPool '**censored**.snowflakecomputing.com', SessionPool 1/1 active sessions
2023-12-15 13:39:01,871 - MainThread network.py:870 - _request_exec_wrapper() - DEBUG - remaining request timeout: 5000 ms, retry cnt: 1
2023-12-15 13:39:01,872 - MainThread network.py:852 - add_request_guid() - DEBUG - Request guid: 63f1f79a-c0ec-4127-aa35-6d1e5f05f506
2023-12-15 13:39:01,872 - MainThread network.py:1060 - _request_exec() - DEBUG - socket timeout: 60
2023-12-15 13:39:01,951 - MainThread connectionpool.py:474 - _make_request() - DEBUG - https://**censored**.snowflakecomputing.com:443 "POST /session?delete=true&request_guid=63f1f79a-c0ec-4127-aa35-6d1e5f05f506 HTTP/1.1" 200 None
2023-12-15 13:39:01,951 - MainThread network.py:1087 - _request_exec() - DEBUG - SUCCESS
2023-12-15 13:39:01,951 - MainThread network.py:1224 - _use_requests_session() - DEBUG - Session status for SessionPool '**censored**.snowflakecomputing.com', SessionPool 0/1 active sessions
2023-12-15 13:39:01,951 - MainThread network.py:740 - _post_request() - DEBUG - ret[code] = None, after post request
2023-12-15 13:39:01,954 - MainThread _query_context_cache.py:141 - clear_cache() - DEBUG - clear_cache() called
2023-12-15 13:39:01,954 - MainThread connection.py:737 - close() - DEBUG - Session is closed

[sfc-gh-aalam]

@jan-who why do you expect the sequence RAW.DWD.TEST to be dropped first, then a new table being created and then renamed to RAW.DWD.TEST?
In terms of behavior, I observed the following

1. create a transient table with set of rows
2. call write_pandas with overwrite=True and auto_create_table=False.
3. expected old values to be gone and table being replaced by news values from write_pandas in step 2

This is the expected behavior. Are you seeing anything different?

[andreqaugusto]

This is a bug that was already reported: #1413

I wrote a reasonably simple fix for it: #1792. However, I'm still waiting for someone to review it.

[jan-who]

@jan-who why do you expect the sequence RAW.DWD.TEST to be dropped first, then a new table being created and then renamed to RAW.DWD.TEST?

Because that is what technically happened. I debugged the sequence of write_pandas() and saw that behavior.

In terms of behavior, I observed the following

1. create a transient table with set of rows
2. call write_pandas with overwrite=True and auto_create_table=False.
3. expected old values to be gone and table being replaced by news values from write_pandas in step 2

This is the expected behavior. Are you seeing anything different?

The expected behavior is a truncate of the table to get rid of the old values, not to drop the table and have a new table created with the same name.

Although the result is pretty much the same, the way to achieve this differ, and in my scenario I do not have "drop table"-rights, but I do have "truncate table"-rights.

Plus the method should technically do what the documentation says it does 🙂

[sfc-gh-aalam]

@jan-who thank you for the explanation.

[sfc-gh-aalam]

fix merged in #1833