Parallel checkpoint of two attached databases deadlocks.

[mima-hlavacek]

What happens?

Doing checkpoints of different attached databases deadlocks. From the documentation I got the sense that checkpoint would be blocking, but in the sense that when checkpointing, all other clients are locked and wait until a given checkpoint finishes.

To Reproduce

from concurrent.futures import ThreadPoolExecutor
from pathlib import Path
from random import random

import duckdb


def run_thread(cursor, name):
    cursor.execute(f"insert into {name}.{name} select sum(i) from range({int(random()*1000000)}) r(i)")
    cursor.execute(f"checkpoint {name}")


Path("foo.duckdb").unlink(missing_ok=True)
Path("bar.duckdb").unlink(missing_ok=True)
conn = duckdb.connect()
cursor_foo = conn.cursor()
cursor_bar = conn.cursor()
cursor_foo.execute("attach 'foo.duckdb' as foo")
cursor_foo.execute("create table foo.foo(foo bigint)")
cursor_bar.execute("attach 'bar.duckdb' as bar")
cursor_bar.execute("create table bar.bar(bar bigint)")

for i in range(1000):
    print(i)
    with ThreadPoolExecutor(max_workers=2) as executor:
        footure = executor.submit(run_thread, cursor_foo, "foo")
        barture = executor.submit(run_thread, cursor_bar, "bar")
        footure.result()
        barture.result()

and wait for a bit.

OS:

Ubuntu x64 in WSL on Windows 11

DuckDB Version:

0.9.2.dev14+g0ef2a6faa2

DuckDB Client:

Python

Full Name:

Míma Hlaváček

Affiliation:

Blindspot.ai

Have you tried this on the latest main branch?

I have tested with a main build

Have you tried the steps to reproduce? Do they include all relevant data and configuration? Does the issue you report still appear there?

• Yes, I have