Poor performance for simple operation on large-ish database.

[jpambrun]

What happens?

I have a large-ish database. I am using libsql and not sqlite proper. It has two tables, one with 25M rows and one with 10M. The db file is ~50GB.

Then

time duckdb data.sqld/dbs/default/data "select * from message_queue limit 10"
real    0m46.187s
user    0m5.659s
sys     0m10.910s

The same query takes 60ms from sqlite.

Even just show tables

time duckdb data.sqld/dbs/default/data "show tables;"
┌─────────────────┐
│      name       │
│     varchar     │
├─────────────────┤
│ dicom           │
│ message_queue   │
│ sqlite_sequence │
└─────────────────┘
real    3m21.493s
user    3m29.433s
sys     0m10.378s

Using SELECT name FROM sqlite_master WHERE type='table'; from a normal client 55ms.

This performance is very surprising. What is going on?

To Reproduce

Can't really share much more.

OS:

linux

SQLite Version:

libsql 0.24.31

DuckDB Version:

v1.1.3 19864453f7

DuckDB Client:

cli

Full Name:

JF Pambrun

Affiliation:

See linkedin..

Have you tried this on the latest main branch?

• I agree

Have you tried the steps to reproduce? Do they include all relevant data and configuration? Does the issue you report still appear there?

• I agree