jupyter deno notebook - ffi issue

[Jean-Baptiste-Lasselle]

~/poc-data-visualization$ docker-compose exec -T jupyter_deno bash -c "source ~/.bashrc && deno --version"
deno 1.40.5 (release, x86_64-unknown-linux-gnu)
v8 12.1.285.27
typescript 5.3.3

Issue Description

Hello awesome deno team, I recently am running a jupyter/deno notebook, inside a container.

I designed the Dockerfile myself, and in my notebook, I try and use an HTML canvas. As I do so, I stumble upon an issue I caan't fathom how to solve.

First, here is the startup script I use as CMD in my docker file:

#!/bin/bash

export PATH="$PATH:$HOME/.cargo/bin"
source ~/.bashrc

pwd

cargo --version


deno --version
deno jupyter --install --unstable-ffi




# tested works # jupyter notebook --unstable true --ip 0.0.0.0 --allow-root
export NOTEBOOKS_DIR=$DATAVIZ_NOTEBOOKS_HOME
jupyter lab --unstable-ffi=true --ip=0.0.0.0 --allow-root --NotebookApp.token='pesto' --NotebookApp.password='pesto' --notebook-dir="$NOTEBOOKS_DIR"

The error I get as I execute my notebook is visible in the webbrowser, and no error shows up in my docker container logs (as you can see, it definitely is the canvas deno which throws the error) :

Note:

• As you can see, I did use, as instructed by the deno documentation, the --unstable option flag.
• In my Dockerfile, I install the kernel with deno jupyter --install --unstable, also as instructed by the deno docs
• That's why, I also added in the startup script above, to even add the deno jupyter --install --unstable-ffi kernel installation command (and i kept the deno jupyter --install --unstable in my Dockerfile): I still get the exact same error
• also note that i do install in my Dockerfile (debian based), the libffi-dev package.
• Finally, i found Deno process does not exit with Promise. #21907 (comment) :
  • In my Dockerfile, I am in a GNU/linux Debian , plus i run the container on a GNU/Linux Debian VM.
  • Does it mean it is not possible to solve my issue, and to use ffi in my notebook, while i run it in a GNU/Linux docker container context ?
  • If so, is there any plan to solve this, I can't afford to ask my users to run their notebooks locally, and only on a windows machine...

The Dockerfile

please forgive me for it is messy, I'm in an early stage:

## ARG BASE_IMAGE=denoland/deno:debian-1.40.5
# inspired by:
#  - https://blog.logrocket.com/deno-jupyter-notebook-data-dashboard/
#  - https://blog.jupyter.org/bringing-modern-javascript-to-the-jupyter-notebook-fc998095081e
#  - https://docs.deno.com/runtime/manual/tools/jupyter
#
# https://hub.docker.com/layers/library/python/3.13.0a4-slim-bookworm/images/sha256-225ac4a4b47349833efc94d61b93c8784279341aff99e714f6ca455ca9232509?context=explore
# ARG BASE_IMAGE=python:3.13.0a4-slim-bookworm
ARG BASE_IMAGE=python:3.10-slim

FROM $BASE_IMAGE as build

ARG BASE_IMAGE=$BASE_IMAGE
ARG DATAVIZ_NOTEBOOKS_HOME=/usr/dataviz/notebooks/decoderleco
ENV DATAVIZ_NOTEBOOKS_HOME=$DATAVIZ_NOTEBOOKS_HOME

RUN apt-get update -y && \
    echo " >>>>>> OK UPDATE COMPLETE " && \
    apt-get install -y curl && \
    echo " >>>>>> OK INSTALL [curl] COMPLETE " && \
    apt-get install -y jq && \
    echo " >>>>>> OK INSTALL [jq] COMPLETE " && \
    apt-get install -y build-essential && \
    echo " >>>>>> OK INSTALL [build-essential] COMPLETE " && \
    apt-get install -y unzip && \
    echo " >>>>>> OK INSTALL [unzip] COMPLETE " && \
    curl -fsSL https://deno.land/install.sh | sh && \
    echo " >>>>>> OK INSTALL [DENO] COMPLETE "
RUN echo " >>>>>> start rust install" && curl https://sh.rustup.rs -sSf | sh -s -- -y && \
    echo " >>>>>> OK INSTALL [RUST +CARGO] COMPLETE "
RUN echo "export PATH=\"\$PATH:\$HOME/.cargo/bin\"" | tee -a ~/.shrc | tee -a ~/.bashrc && bash -c 'source "$HOME/.cargo/env"'
RUN ls $HOME/.cargo/bin
RUN export PATH="$PATH:$HOME/.cargo/bin" && cargo --version
RUN apt-get install -y build-essential libssl-dev libffi-dev libzmq3-dev
# >> reparations
# RUN apt-get install -y build-essential libssl-dev libffi-dev libzmq3-dev && apt-get install -y --fix-broken
RUN apt-get install -y build-essential libssl-dev libffi-dev libzmq3-dev

# RUN export PATH="$PATH:$HOME/.cargo/bin" && export PIP_DEFAULT_TIMEOUT=200 && python --version && python -m pip install pyzmq --trusted-host pypi.org --trusted-host files.pythonhosted.org

WORKDIR /app
COPY requirements.txt /app
# RUN pip install -r requirements.txt
RUN python -m pip install --upgrade pip
RUN export PATH="$PATH:$HOME/.cargo/bin" && export PIP_DEFAULT_TIMEOUT=200 && pip install -r requirements.txt

RUN export PATH="$PATH:$HOME/.cargo/bin" && export PIP_DEFAULT_TIMEOUT=200 && pip install jupyterlab notebook && \
    echo " >>>>>> OK INSTALL [pip install jupyterlab notebook] COMPLETE "
RUN curl -fsSL https://deno.land/install.sh | sh && \
    echo 'export DENO_INSTALL="/root/.deno"' | tee -a ~/.bashrc | tee -a ~/.shrc && \
    echo 'export PATH="$DENO_INSTALL/bin:$PATH"' | tee -a ~/.bashrc | tee -a ~/.shrc && \
    echo " >>>>>> OK INSTALL [DENO] COMPLETE "
RUN bash -c 'source ~/.bashrc && deno jupyter --unstable --install' && \
    echo " >>>>>> OK INSTALL [deno jupyter --unstable --install] COMPLETE "

RUN mkdir -p $DATAVIZ_NOTEBOOKS_HOME


FROM build as runner

RUN mkdir -p /run


COPY start.sh .
RUN chmod +x ./start.sh && mv ./start.sh /run

# CMD [ "jupyter", "notebook" ]

WORKDIR /usr/dataviz/notebooks/decoderleco
EXPOSE 8888
# CMD [ "jupyter", "notebook", "--unstable" ]
CMD [ "/run/start.sh" ]

[sergiunagy]

I have the same issue when using Deno from the Ubuntu Docker image, [denoland/deno:ubuntu-1.41.0]. Is there some reason you don't use their official docker images (https://hub.docker.com/r/denoland/deno) ? I'm new to Deno and I'm asking to see if I missed something.

In my case this fails with the same message when trying to import from skia_canvas.

I see on their repo the make an explicit note to use specific flags allowing unstable ffi content but I could not use that with Jupyter.
https://github.com/DjDeveloperr/skia_canvas?tab=readme-ov-file#usage

I also see that the Deno-Jupyter-D3 combination has been working in the past since this code is present in multiple example articles.

[Jean-Baptiste-Lasselle]

articles

Hello @sergiunagy thank you for your message:

• I do not use the official ubuntu image, because i always prefer debian for deeply technical things: the debian community is more serious in their publications, usually, and for all documentation. Also Debian is very sttable and solid globally
• As of today, I still have the exact same issue: I am not yet able to say why the ffi lib feature wouldn't work even though I tried all usage of the unstable ffi flag I could think of yet.
• the only thing I could thinkof is that there might be a lack of linux packages underneath: as you can see in my docker image, i tried many combinations, yet for now i couldn't make it work:
  • my most natural idea would be to start from a deno official image, where unstable ffi flags should work
  • yet it's a hassle to properly intallthe full python stack required by jupyter: so its gonna take time, before i complete taht, and i have other responsibilities in my project, so well, if you have news i'm really a taker...

[Jean-Baptiste-Lasselle]

I have the same issue when using Deno from the Ubuntu Docker image, [denoland/deno:ubuntu-1.41.0]. Is there some reason you don't use their official docker images (https://hub.docker.com/r/denoland/deno) ? I'm new to Deno and I'm asking to see if I missed something.

In my case this fails with the same message when trying to import from skia_canvas.

I see on their repo the make an explicit note to use specific flags allowing unstable ffi content but I could not use that with Jupyter. https://github.com/DjDeveloperr/skia_canvas?tab=readme-ov-file#usage

I also see that the Deno-Jupyter-D3 combination has been working in the past since this code is present in multiple example articles.

add on : ouh oh my, you must pass --allow-env, --allow-ffi and --unstable-ffi flags , i will try using together all 3, i haven't tested that yet, thanks for the ref @sergiunagy !

[Jean-Baptiste-Lasselle]

I have the same issue when using Deno from the Ubuntu Docker image, [denoland/deno:ubuntu-1.41.0]. Is there some reason you don't use their official docker images (https://hub.docker.com/r/denoland/deno) ? I'm new to Deno and I'm asking to see if I missed something.

In my case this fails with the same message when trying to import from skia_canvas.

I see on their repo the make an explicit note to use specific flags allowing unstable ffi content but I could not use that with Jupyter. https://github.com/DjDeveloperr/skia_canvas?tab=readme-ov-file#usage

I also see that the Deno-Jupyter-D3 combination has been working in the past since this code is present in multiple example articles.

Hello again @sergiunagy , a bit more to answer you:

• first, in the denoland/deno you mention, the typescript/deno source code you talk about is executed directly by deno. But in my case, I want to execute the same source code (involving skia_canvas), but in a jupyter notebook, with a deno kernel. In denoland/deno, there is nothing related to jupyter at all, it is just a bare deno runtime.
• Yet, thank yiou for yoru feedback: it makes sense, that the deno runtime, whether bare, or as a jupyter kernel, spits the same error, about this skia_canvas source code we are interested in.
• Starting the jupyter app, inside a container, is done running a jupyter command, would it make sense to feed the jupyter command, deno GNU options? Well, you can see below a few tests I have performed, all ended up either jupyter failing to start, or the skaia_canvas causing the same ffi related error :

#!/bin/bash

export PATH="$PATH:$HOME/.cargo/bin"
source ~/.bashrc

pwd

cargo --version


deno --version
# KO # deno jupyter --install --unstable-ffi --allow-ffi --allow-env
# KO # deno jupyter --install --unstable-ffi --allow-env
# OK but does not fix [ffi dl open] error # deno jupyter --install --unstable-ffi --unstable --unstable-fs --unstable-worker-options
deno jupyter --install --unstable-ffi --unstable-fs # without [--unstable] which is deprecated for [--unstable-*] options
# deno jupyter --install --unstable-ffi

# tested works # jupyter notebook --unstable true --ip 0.0.0.0 --allow-root
export NOTEBOOKS_DIR=$DATAVIZ_NOTEBOOKS_HOME
# OK # jupyter lab --unstable=true --ip=0.0.0.0 --allow-root --NotebookApp.token='decoderleco' --NotebookApp.password='decoderleco' --notebook-dir="$NOTEBOOKS_DIR"
# KO # jupyter-lab: error: unrecognized arguments: --unstable-ffi=true --unstable-fs=true # jupyter lab --unstable=true --unstable-ffi=true --unstable-fs=true --ip=0.0.0.0 --allow-root --NotebookApp.token='decoderleco' --NotebookApp.password='decoderleco' --notebook-dir="$NOTEBOOKS_DIR"
# KO # jupyter-lab: error: unrecognized arguments: --unstable-ffi --unstable-fs # jupyter lab --unstable=true --unstable-ffi --unstable-fs --ip=0.0.0.0 --allow-root --NotebookApp.token='decoderleco' --NotebookApp.password='decoderleco' --notebook-dir="$NOTEBOOKS_DIR"
jupyter lab --unstable=true --ip=0.0.0.0 --allow-root --NotebookApp.token='decoderleco' --NotebookApp.password='decoderleco' --notebook-dir="$NOTEBOOKS_DIR"

Finally, I could do some more tests, finding out how one can configure jupyter to pass arguments and options to a jupyter kernel. But if it does not work with a bare deno, there is not one chance it would as a jupyter kernel: so trygin tests with a bare deno runtime is lighter, ad faster.

Now the question is : what more tests can we do ?

I say this:

• first lets not forget what --unstable-ffi: it means this ffi feature might disappear anytime in future versions of deno. This also means that iff it does not work with todays latest version of deno, it certainly was working in a previous version of deno : at the date the article/blog posts were pulblished, what was the latest version of deno ...?
• the only one other parameter we can play with in tests, is the following: the linux packages installed "under" the deno installation: things like libffi-dev and a few others, might be required, and are missing in some containers

WDYT?

Could we join efforts to design a whole set of tests and catch this animal together ? the question and reached hand is meant at all readers as well.

ps:

For the reference:

• FFI shared lib imports #15674

[lucacasonato]

It does indeed seem like there is currently no way to configure the Jupyter Kernel with --unstable flags - I'll mark this as a bug.