bug: connecting to MotherDuck broken on main

[lostmygithubaccount]

What happened?

I first noticed this in the Ibis analytics streamlit app, with an error eventually at:

  File "/home/adminuser/venv/lib/python3.11/site-packages/ibis/backends/duckdb/__init__.py", line 441, in do_connect

    self.con = duckdb.connect(str(database), config=config, read_only=read_only)

               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

duckdb.duckdb.CatalogException: Catalog Error: Cannot open database "/mount/src/ibis-analytics/md:ibis_analytics" in read-only mode: database does not exist

after some trial and error, I could reproduce this locally:

import ibis

con = ibis.connect("duckdb://md:ibis_analytics", read_only=True)
con

leading to the same error, full traceback below. making read_only=False creates the md:ibis_analytics duckdb files in the working directory as it would for a normal database

I'm guessing motherduck parsing for the URI got dropped somewhere in the refactor?

What version of ibis are you using?

main

What backend(s) are you using, if any?

duckdb==0.9.2

Relevant log output

---------------------------------------------------------------------------
CatalogException                          Traceback (most recent call last)
Cell In[3], [line 1](vscode-notebook-cell:?execution_count=3&line=1)
----> [1](vscode-notebook-cell:?execution_count=3&line=1) con = ibis.connect("duckdb://md:ibis_analytics", read_only=True)
      [2](vscode-notebook-cell:?execution_count=3&line=2) con

File [~/repos/ibis/ibis/backends/base/__init__.py:1407](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/base/__init__.py:1407), in connect(resource, **kwargs)
   [1404](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/base/__init__.py:1404) except AttributeError:
   [1405](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/base/__init__.py:1405)     raise ValueError(f"Don't know how to connect to {resource!r}") from None
-> [1407](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/base/__init__.py:1407) return backend._from_url(url, **orig_kwargs)

File [~/repos/ibis/ibis/backends/base/__init__.py:1446](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/base/__init__.py:1446), in UrlFromPath._from_url(self, url, **kwargs)
   [1443](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/base/__init__.py:1443)         raise exc.IbisError(f"Invalid URL parameter: {name}")
   [1445](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/base/__init__.py:1445) self._convert_kwargs(kwargs)
-> [1446](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/base/__init__.py:1446) return self.connect(database=database, **kwargs)

File [~/repos/ibis/ibis/backends/base/__init__.py:864](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/base/__init__.py:864), in BaseBackend.connect(self, *args, **kwargs)
    [840](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/base/__init__.py:840) """Connect to the database.
    [841](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/base/__init__.py:841) 
    [842](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/base/__init__.py:842) Parameters
   (...)
    [861](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/base/__init__.py:861) 
    [862](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/base/__init__.py:862) """
    [863](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/base/__init__.py:863) new_backend = self.__class__(*args, **kwargs)
--> [864](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/base/__init__.py:864) new_backend.reconnect()
    [865](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/base/__init__.py:865) return new_backend

File [~/repos/ibis/ibis/backends/base/__init__.py:874](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/base/__init__.py:874), in BaseBackend.reconnect(self)
    [872](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/base/__init__.py:872) def reconnect(self) -> None:
    [873](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/base/__init__.py:873)     """Reconnect to the database already configured with connect."""
--> [874](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/base/__init__.py:874)     self.do_connect(*self._con_args, **self._con_kwargs)

File [~/repos/ibis/ibis/backends/duckdb/__init__.py:441](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/duckdb/__init__.py:441), in Backend.do_connect(self, database, read_only, temp_directory, extensions, **config)
    [438](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/duckdb/__init__.py:438)     Path(temp_directory).mkdir(parents=True, exist_ok=True)
    [439](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/duckdb/__init__.py:439)     config["temp_directory"] = str(temp_directory)
--> [441](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/duckdb/__init__.py:441) self.con = duckdb.connect(str(database), config=config, read_only=read_only)
    [443](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/duckdb/__init__.py:443) # Load any pre-specified extensions
    [444](https://file+.vscode-resource.vscode-cdn.net/Users/cody/repos/ibis/~/repos/ibis/ibis/backends/duckdb/__init__.py:444) if extensions is not None:

CatalogException: Catalog Error: Cannot open database "/Users/cody/repos/ibis/md:ibis_analytics" in read-only mode: database does not exist

Code of Conduct

• I agree to follow this project's Code of Conduct

[gforsyth]

We still have that code, but DuckDB appears to have a different way of handling this now?

[ins] In [3]: import duckdb

[ins] In [4]: duckdb.__version__
Out[4]: '0.9.2'

[ins] In [5]: con = duckdb.connect("motherduck://")
---------------------------------------------------------------------------
IOException                               Traceback (most recent call last)
Cell In[5], line 1
----> 1 con = duckdb.connect("motherduck://")

IOException: IO Error: Cannot open file "/home/gil/github.com/ibis-project/ibis/motherduck://": Is a directory

[gforsyth]

oh, hilarious, it can't have the trailing slashes

[cpcloud]

I think we're probably doing something incorrect with the URL. This code did change in The Great Port

[gforsyth]

ugh, I forgot that motherduck can't handle DNSoverTLS. I can fix this up (the ibis issue, I can't make motherduck support modern dns)

[gforsyth]

Hmm, @lostmygithubaccount can you try doing a FORCE install motherduck; in duckdb?

after making my internet less secure, I can still connect:

[ins] In [1]: import ibis

[ins] In [2]: con = ibis.duckdb.connect("md:my_db")

[ins] In [3]: con.list_tables()
Out[3]: ['penguins']

[ins] In [4]: con = ibis.duckdb.connect("motherduck:my_db")

[ins] In [5]: con.list_tables()
Out[5]: ['penguins']

[lostmygithubaccount]

that works (ibis.duckdb.connect) but the ibis.connect("duckdb://md:") (or with motherduck:) still does not (and used to)

[gforsyth]

OOOOOH, gotcha