IO Error: buffer_ptr not set in CheckHeader

[cobolbaby]

What happens?

Using DuckDB to export a PostgreSQL table to a Parquet file, if the table in PostgreSQL is a view, you may encounter the following error IO Error: buffer_ptr not set in CheckHeader. I haven't figured out the exact reason yet.

To Reproduce

COPY (SELECT * FROM {POSTGRES_DB}.{POSTGRES_SCHEMA}.{table_name}) TO '{table_dump_file}' (FORMAT PARQUET, COMPRESSION ZSTD, ROW_GROUP_SIZE 100000);

OS:

x86_64

DuckDB Version:

1.1.2

DuckDB Client:

Python

Hardware:

No response

Full Name:

Cobol

Affiliation:

Inventec

What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

I have not tested with any build

Did you include all relevant data sets for reproducing the issue?

No - Other reason (please specify in the issue body)

Did you include all code required to reproduce the issue?

• Yes, I have

Did you include all relevant configuration (e.g., CPU architecture, Python version, Linux distribution) to reproduce the issue?

• Yes, I have

[moritz93]

I get the same error using duckdb CLI v1.0.0 when reading from a Timescale hypertable.

[naoyak]

Here to report the same thing, also with a Timescale table.

[moritz93]

In my case the errors disappeared after setting sane values for PG config parameters:

max_standby_streaming_delay
max_standby_archive_delay

So the errors might come from PG terminating the COPY connection when it is taking too long in respect to the above parameters.

[Mytherin]

Thanks for reporting!

I've managed to reproduce this issue by setting a low statement timeout and running a query that triggers a lot of work in the server:

ATTACH 'dbname=postgresscanner options=''-c statement_timeout=10000''' AS s (TYPE POSTGRES, READ_ONLY);
SELECT * FROM postgres_query(s, 'select count(*) from tbl, tbl t2');
-- IO Error: buffer_ptr not set in CheckHeader

As indicated this is not really a problem we can fix as it is a Postgres configuration problem - but I'll rework the error message to be more descriptive.