Support for project-level config files

[StefanBrand]

Elevator Pitch

Filtering the relevant diagnostics should be as easy as git cloneing a repo with a project-level setup.cfg.

Motivation

We are working collaboratively on a repo of Jupyter notebooks. When working on a Python project, we are accustomed to have a setup.cfg that configures the linters for the whole project. The same convenience should be in place for this repo of notebooks.

Design Ideas

A difficulty that I see is that at the moment jupyter-lsp does not seem to have a concept of "project boundary". Therefore it might be an issue to find project-level configuration files.

Possible solutions:

• simply walk up the file tree a number of levels
• use git rev-parse --show-toplevel if available and find config files there and in any sub-directories

[krassowski]

I am not convinced that we should be doing this though.

I think that this should be addressed by specific servers like python-lsp-server (pylsp) or jedi-language-server instead. The LSP protocol does not support language-specific configuration files like setup.cfg but individual server can and do - for example pylsp already does it by reading:

• .flake8, setup.cfg, and tox.ini for flake8
• pycodestyle.cfg, setup.cfg, and tox.ini for pycodestyle

Our settings for filtering diagnostics are meant to allow tool-agnostic approach for servers that do not support such tools-specific configuration. I would prefer to avoid creating a new standard and advise to help extending the language servers instead so that they respect more configuration from setup.cfg. What do you think?

[StefanBrand]

I did manage to configure flake8 in my home directory, but I could not get it to work with a setup.cfg directly in the project folder structure, e.g.

repository
|_ setup.cfg
|_ Untitled.ipynb

The setup.cfg would not apply to the notebook. If placed in ~/setup.cfg it applies immediately after reopening the notebook.

Now, I have no idea about the details of the architecture between this project and the language servers. I see that the Python language server discovers configs using a find_parents function, but I do not observe that it works.

If this is the wrong place to discuss this, could you point me to the correct repository, please? :)

[krassowski]

I would such placement of setup.cfg to work. Feel free to open an issue in https://github.com/python-lsp/python-lsp-server; if you feel comfortable with Python you might be the best person to debug this at the moment, I don't believe any one else touched this code recently.

[krassowski]

Re-opening; the issue was tracked down to the setup.cfg being placed outside of the JuypterLab root in python-lsp/python-lsp-server#54 (comment). Placing it into a sub-directory does not work because of .virtual_documents approach to presenting files from disk to the server.

We could workaround this by placing the dummy python files in their true paths but as hidden files. Pros:

• no relative paths issues
• setup.cfg can be anywhere

Cons:

• need to do less-trivial path transformations (so far we just strip .virtual_documents prefix)
• may be annoying for version control systems

[StefanBrand]

may be annoying for version control systems

Many tools put files/directories into the project. E.g. our .gitignore has these entries:

.ipynb_checkpoints
.idea
.vscode

I would even go so far as to say that all projects with Jupyter notebooks have .ipynb_checkpoints in their .gitignore files. 😄 So from my perspective an additional directory would not hurt.

[krassowski]

Yes, but I am not proposing to use a single folder (we already do that) but one hidden file for each notebook, in the same folder as is the notebook. Are you really sure you cannot change the startup directory for JupyterHub?

A simpler solution might be to fix python-lsp-server to operate on in-memory files rather than on disk (as it should do according to the LSP protocol), but this is at mercy of other maintainers agreeing to such a rewrite of multiple plugins without clear advantages to their projects...

[StefanBrand]

Are you really sure you cannot change the startup directory for JupyterHub?

Technically it is certainly possible, but that single directory would be the new default for all users on our JupyterHub. It's not a good idea to change the home directory for all users. 😄 Besides, it would only work for one project and not for multiple project's with their own setup.cfg.

[StefanBrand]

Hidden Python files in the project root could be excluded in .gitignore like this:/.*.py

[krassowski]

Out of curiosity what would happen if you create ~/.jupyter/jupyter_notebook_config.py or ~/.jupyter/jupyter_server_config.py with:

import os
c.ServerApp.root_dir = os.path.expanduser('~/test')

I would hope that this should only affect you as the user as it is not modifying the Spawner.

[krassowski]

The next version of jupyter_server will also have preferred_dir: jupyter-server/jupyter_server#549

[StefanBrand]

Out of curiosity what would happen if you create ~/.jupyter/jupyter_notebook_config.py or ~/.jupyter/jupyter_server_config.py with:

I tried both config files and it does not seem to change anything. The directory that appears when JupyterLab is ready is still /home/jovyan.