Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Native server does not work with Jupyter Notebook without configuration #546

Closed
Xiao-Chenguang opened this issue Jul 21, 2024 · 16 comments
Closed
Assignees
Labels
native-server Related to the native server notebook Related to (Jupyter) notebooks

Comments

@Xiao-Chenguang
Copy link

Xiao-Chenguang commented Jul 21, 2024

Ruff not function in jupyter notebooks when set ruff.nativeServer to auto or on. It seems there are some communication problem between the nativeserver and notebook. When use ruff-lsp by setting ruff.nativeServer to off, all ruff functions works as expected.

According to the server log, the nativeserver started normally but receive nothing while I type format command.

2024-07-21 16:44:25.149 [info] Name: Ruff
2024-07-21 16:44:25.149 [info] Module: ruff
2024-07-21 16:44:25.149 [info] Python extension loading
2024-07-21 16:44:25.149 [info] Waiting for interpreter from python extension.
2024-07-21 16:44:25.149 [info] Python extension loaded
2024-07-21 16:44:25.242 [info] Falling back to bundled executable: /Users/cg/.vscode/extensions/charliermarsh.ruff-2024.34.0-darwin-arm64/bundled/libs/bin/ruff
2024-07-21 16:44:25.265 [info] Resolved 'ruff.nativeServer: auto' to use the native server
2024-07-21 16:44:25.266 [info] Found Ruff 0.5.3 at /Users/cg/.vscode/extensions/charliermarsh.ruff-2024.34.0-darwin-arm64/bundled/libs/bin/ruff
2024-07-21 16:44:25.266 [info] Server run command: /Users/cg/.vscode/extensions/charliermarsh.ruff-2024.34.0-darwin-arm64/bundled/libs/bin/ruff server
2024-07-21 16:44:25.266 [info] Server: Start requested.

The behaviour of setting nativeServer off is:

2024-07-21 16:48:31.470 [info] Name: Ruff
2024-07-21 16:48:31.470 [info] Module: ruff
2024-07-21 16:48:31.470 [info] Python extension loading
2024-07-21 16:48:31.470 [info] Waiting for interpreter from python extension.
2024-07-21 16:48:31.470 [info] Python extension loaded
2024-07-21 16:48:31.470 [info] Server run command: /Users/cg/miniconda3/envs/fl/bin/python /Users/cg/.vscode/extensions/charliermarsh.ruff-2024.34.0-darwin-arm64/bundled/tool/server.py
2024-07-21 16:48:31.470 [info] Server: Start requested.
2024-07-21 16:48:31.934 [info] 2024-07-21 16:48:31,934 INFO Starting IO server

2024-07-21 16:48:31.975 [info] Workspace settings: []
2024-07-21 16:48:31.975 [info] Global settings: {
    "nativeServer": "off",
    "cwd": "/",
    "workspace": "/",
    "path": [],
    "ignoreStandardLibrary": true,
    "interpreter": [],
    "configuration": null,
    "importStrategy": "fromEnvironment",
    "codeAction": {
        "fixViolation": {
            "enable": true
        },
        "disableRuleComment": {
            "enable": true
        }
    },
    "lint": {
        "enable": true,
        "run": "onType",
        "args": []
    },
    "format": {
        "args": []
    },
    "enable": true,
    "organizeImports": true,
    "fixAll": true,
    "showNotifications": "off",
    "configurationPreference": "editorFirst",
    "showSyntaxErrors": true
}
2024-07-21 16:48:31.990 [info] Interpreter executable (/Users/cg/miniconda3/envs/fl/bin/ruff) not found
2024-07-21 16:48:31.990 [info] Falling back to bundled executable: /Users/cg/.vscode/extensions/charliermarsh.ruff-2024.34.0-darwin-arm64/bundled/libs/bin/ruff
2024-07-21 16:48:32.025 [info] Inferred version 0.5.3 for: /Users/cg/.vscode/extensions/charliermarsh.ruff-2024.34.0-darwin-arm64/bundled/libs/bin/ruff
2024-07-21 16:48:32.025 [info] Found ruff 0.5.3 at /Users/cg/.vscode/extensions/charliermarsh.ruff-2024.34.0-darwin-arm64/bundled/libs/bin/ruff
2024-07-21 16:48:32.025 [info] Running Ruff with: /Users/cg/.vscode/extensions/charliermarsh.ruff-2024.34.0-darwin-arm64/bundled/libs/bin/ruff ['check', '--force-exclude', '--no-cache', '--no-fix', '--quiet', '--output-format', 'json', '-', '--stdin-filename', '/Users/cg/Desktop/test.ipynb']

other ruff setting: all default
ruff version: v2024.34.0
OS: Mac
Python version: 3.10

Mininal example:

# test.ipynb
a = [[1, 2, 3, 4, 5], [6, 7, 8, 9, 10], [11, 12, 13, 14, 15], [16, 17, 18, 19, 20], [21, 22, 23, 24, 25]]
@charliermarsh
Copy link
Member

\cc @dhruvmanila

@dhruvmanila
Copy link
Member

Can you provide the contents of the Ruff config in pyproject.toml / ruff.toml / .ruff.toml file? After astral-sh/ruff#12252, Ruff would only include the notebook files if it's included via either include or extend-include.

The reason it works for ruff-lsp is because it uses the ruff CLI directly via stdin and it would always provide diagnostics / formatting for a file which is passed directly:

2024-07-22 10:08:25.812 [info] Running Ruff with: /Users/dhruv/work/astral/ruff/target/debug/ruff ['check', '--force-exclude', '--no-cache', '--no-fix', '--quiet', '--output-format', 'json', '-', '--stdin-filename', '/Users/dhruv/playground/ruff/notebooks/test.ipynb']

This doesn't consider the include / extend-include config because the --force-exclude flag doesn't consider those settings.

@dhruvmanila
Copy link
Member

You can also enabled debug logs for the native server using:

{
  "ruff.trace.server": "messages",
  "ruff.logLevel": "debug"
}

which outputs:

2024-07-22 10:03:54.657 [info] Using 'path' setting: /Users/dhruv/work/astral/ruff/target/debug/ruff
2024-07-22 10:03:55.471 [info] Found Ruff 0.5.2 at /Users/dhruv/work/astral/ruff/target/debug/ruff
2024-07-22 10:03:55.471 [info] Server run command: /Users/dhruv/work/astral/ruff/target/debug/ruff server --preview
2024-07-22 10:03:55.472 [info] Server: Start requested.
2024-07-22 10:03:55.476 [info] [Trace - 10:03:55 AM] Sending request 'initialize - (0)'.
2024-07-22 10:03:55.488 [info] [Trace - 10:03:55 AM] Received response 'initialize - (0)' in 12ms.
2024-07-22 10:03:55.489 [info] [Trace - 10:03:55 AM] Sending notification 'initialized'.
2024-07-22 10:03:55.490 [info] [Trace - 10:03:55 AM] Sending notification 'textDocument/didOpen'.
2024-07-22 10:03:55.490 [info] [Trace - 10:03:55 AM] Sending request 'textDocument/diagnostic - (1)'.
2024-07-22 10:03:55.490 [info] [Trace - 10:03:55 AM] Sending notification 'notebookDocument/didOpen'.
2024-07-22 10:03:55.492 [info] [Trace - 10:03:55 AM] Sending request 'textDocument/codeAction - (2)'.
2024-07-22 10:03:55.493 [info] [Trace - 10:03:55 AM] Sending request 'textDocument/codeAction - (3)'.
2024-07-22 10:03:55.494 [info] [Trace - 10:03:55 AM] Sending request 'textDocument/codeAction - (4)'.
2024-07-22 10:03:55.510 [info]    0.019833583s DEBUG ThreadId(13) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/playground/ruff/.vscode

2024-07-22 10:03:55.519 [info] [Trace - 10:03:55 AM] Received request 'client/registerCapability - (1)'.
2024-07-22 10:03:55.519 [info] [Trace - 10:03:55 AM] Sending response 'client/registerCapability - (1)'. Processing request took 1ms
2024-07-22 10:03:55.519 [info]    0.028470583s DEBUG ruff:worker:0 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/src/play.py

2024-07-22 10:03:55.520 [info]    0.029595292s  INFO     ruff:main ruff_server::server: Configuration file watcher successfully registered

2024-07-22 10:03:55.520 [info] [Trace - 10:03:55 AM] Received response 'textDocument/codeAction - (4)' in 26ms.

You can see that it only included the Python file (search for "Included path via") and not the notebook file.

But, once you include it in the config:

[tool.ruff]
extend-include = ["*.ipynb"]

It'll start including the notebook files as well.

@Xiao-Chenguang
Copy link
Author

You can also enabled debug logs for the native server using:

{
  "ruff.trace.server": "messages",
  "ruff.logLevel": "debug"
}

which outputs:

2024-07-22 10:03:54.657 [info] Using 'path' setting: /Users/dhruv/work/astral/ruff/target/debug/ruff
2024-07-22 10:03:55.471 [info] Found Ruff 0.5.2 at /Users/dhruv/work/astral/ruff/target/debug/ruff
2024-07-22 10:03:55.471 [info] Server run command: /Users/dhruv/work/astral/ruff/target/debug/ruff server --preview
2024-07-22 10:03:55.472 [info] Server: Start requested.
2024-07-22 10:03:55.476 [info] [Trace - 10:03:55 AM] Sending request 'initialize - (0)'.
2024-07-22 10:03:55.488 [info] [Trace - 10:03:55 AM] Received response 'initialize - (0)' in 12ms.
2024-07-22 10:03:55.489 [info] [Trace - 10:03:55 AM] Sending notification 'initialized'.
2024-07-22 10:03:55.490 [info] [Trace - 10:03:55 AM] Sending notification 'textDocument/didOpen'.
2024-07-22 10:03:55.490 [info] [Trace - 10:03:55 AM] Sending request 'textDocument/diagnostic - (1)'.
2024-07-22 10:03:55.490 [info] [Trace - 10:03:55 AM] Sending notification 'notebookDocument/didOpen'.
2024-07-22 10:03:55.492 [info] [Trace - 10:03:55 AM] Sending request 'textDocument/codeAction - (2)'.
2024-07-22 10:03:55.493 [info] [Trace - 10:03:55 AM] Sending request 'textDocument/codeAction - (3)'.
2024-07-22 10:03:55.494 [info] [Trace - 10:03:55 AM] Sending request 'textDocument/codeAction - (4)'.
2024-07-22 10:03:55.510 [info]    0.019833583s DEBUG ThreadId(13) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/playground/ruff/.vscode

2024-07-22 10:03:55.519 [info] [Trace - 10:03:55 AM] Received request 'client/registerCapability - (1)'.
2024-07-22 10:03:55.519 [info] [Trace - 10:03:55 AM] Sending response 'client/registerCapability - (1)'. Processing request took 1ms
2024-07-22 10:03:55.519 [info]    0.028470583s DEBUG ruff:worker:0 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/src/play.py

2024-07-22 10:03:55.520 [info]    0.029595292s  INFO     ruff:main ruff_server::server: Configuration file watcher successfully registered

2024-07-22 10:03:55.520 [info] [Trace - 10:03:55 AM] Received response 'textDocument/codeAction - (4)' in 26ms.

You can see that it only included the Python file (search for "Included path via") and not the notebook file.

But, once you include it in the config:

[tool.ruff]
extend-include = ["*.ipynb"]

It'll start including the notebook files as well.

Thank you @dhruvmanila , setting extend-include = ["*.ipynb"] in a ruff.toml file works for me. I didn't use ruff config file before and I would suggest you put this as a notice at some place obvious.

@juanitorduz
Copy link

Thanks for the tip! I was having the same problem and

[tool.ruff]
extend-include = ["*.ipynb"]

solved it for me as well!

@dhruvmanila
Copy link
Member

No worries. It makes sense for this change to be documented. I'm thinking of adding it to:

  1. https://github.com/astral-sh/ruff-vscode#jupyter-notebook-support
  2. https://docs.astral.sh/ruff/editors/features/#jupyter-notebook

Do you have any other places in mind which will be more visible?

juanitorduz added a commit to pymc-labs/pymc-marketing that referenced this issue Jul 22, 2024
wd60622 pushed a commit to pymc-labs/pymc-marketing that referenced this issue Jul 22, 2024
* make nb ruff astral-sh/ruff-vscode#546

* fix docs

* fix quickstart

* add types
@Xiao-Chenguang
Copy link
Author

That's really helpful. Use it as an extension, I would first check the extension readme as you mentioned and then the extension setting. It may benefit users by adding this into extension setting and the notebook feature can be enabled global without need of toml file per project.

@dhruvmanila
Copy link
Member

It may benefit users by adding this into extension setting and the notebook feature can be enabled global without need of toml file per project.

Yeah, I had a similar thought earlier but that would mean that running Ruff from the command-line would give different results compared to the editor. I'm not sure if there are any opinions on this but as the notebook files were always included for linting and formatting via ruff-lsp and there haven't been any user report, I guess that should be fine (?).

@Xiao-Chenguang
Copy link
Author

I'm not sure if there are any opinions on this but as the notebook files were always included for linting and formatting via ruff-lsp and there haven't been any user report, I guess that should be fine (?).

Totally agree. Just keep the notebook behaviour similar to ruff-lsp before there is a better solution.

dhruvmanila added a commit to astral-sh/ruff that referenced this issue Jul 22, 2024
## Summary

Similar to astral-sh/ruff-vscode#547 but for the
online docs.

Refer to astral-sh/ruff-vscode#546

## Preview

<img width="1728" alt="Screenshot 2024-07-22 at 14 51 40"
src="https://github.com/user-attachments/assets/39014278-c868-45b0-9058-42858a060fd8">
dhruvmanila added a commit that referenced this issue Jul 22, 2024
## Summary

This PR adds a new section in the README to guide users to opt-in for
Jupyter Notebook support for the native server. This is required
because, unlike `ruff-lsp`, the native server looks at the `include` /
`extend-include` setting.

Refer #546
@dhruvmanila
Copy link
Member

We might just include Jupyter Notebooks in Ruff's inclusion by default (astral-sh/ruff#12456).

@dhruvmanila dhruvmanila added the notebook Related to (Jupyter) notebooks label Jul 23, 2024
@dhruvmanila dhruvmanila changed the title nativeServer not work with jupyter notebook Native server does not work with Jupyter Notebook without configuration Jul 23, 2024
@dhruvmanila dhruvmanila added the native-server Related to the native server label Jul 23, 2024
@rnckp
Copy link

rnckp commented Jul 26, 2024

@dhruvmanila

I still have difficulties making it work, even when I add extend-include = ["*.ipynb"] in ~.vscode/extensions/charliermarsh.ruff-2024.36.0-darwin-arm64/pyproject.toml.

.py files in VS Code are being formatted and linted properly. However, jupyer notebooks are not.

Tried to restart, reinstall from clean state (VS Code, settings and extensions) to no avail. Any recommendations to make it work again? I noticed that I stopped working a couple of days ago.

I use all latest versions of VS Code and extensions. Besides Ruff I have installed Python, Pylance, Python Debugger (all MS), Jupyter, Jupyter Keymap and Jupyter Notebook Renderers.

@dhruvmanila
Copy link
Member

I still have difficulties making it work, even when I add extend-include = ["*.ipynb"] in ~.vscode/extensions/charliermarsh.ruff-2024.36.0-darwin-arm64/pyproject.toml.

I think you'd need to add that in the pyproject.toml file within the current project you're in. That directory is where the extension resides. For example, if you opened VS Code in ~/projects/ruff, then the root config file would be ~/projects/ruff/pyproject.toml.

@rnckp
Copy link

rnckp commented Jul 26, 2024

That did the trick! Thanks so much! You saved my day... 🙏☀️

A detail: In my case it was sufficient to create pyproject.toml just in the root folder of my project (aka the top code folder that I opened in my VS Code workspace). Subfolder /ruff/ didn't seem to be necessary.

@dhruvmanila
Copy link
Member

~/projects/ruff is the root folder of the "ruff" project ;)

@rnckp
Copy link

rnckp commented Jul 26, 2024

Got it! Thanks! 😊

@DGrady
Copy link

DGrady commented Jul 31, 2024

We might just include Jupyter Notebooks in Ruff's inclusion by default (astral-sh/ruff#12456).

Upvoting this! Speaking as someone who mainly uses Ruff from within Jupyter notebooks running in VSCode, my experience here was: a few months ago, very happy that I got to replace two slow extensions with one fast one; then recent frustration when the fast one stopped working with no obvious errors.

AlexWaygood added a commit to astral-sh/ruff that referenced this issue Aug 14, 2024
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
Closes: #12456
Closes: astral-sh/ruff-vscode#546
MichaReiser pushed a commit to astral-sh/ruff that referenced this issue Aug 14, 2024
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
Closes: #12456
Closes: astral-sh/ruff-vscode#546
MichaReiser pushed a commit to astral-sh/ruff that referenced this issue Aug 14, 2024
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
Closes: #12456
Closes: astral-sh/ruff-vscode#546
twiecki pushed a commit to pymc-labs/pymc-marketing that referenced this issue Sep 10, 2024
* make nb ruff astral-sh/ruff-vscode#546

* fix docs

* fix quickstart

* add types
twiecki pushed a commit to pymc-labs/pymc-marketing that referenced this issue Sep 10, 2024
* make nb ruff astral-sh/ruff-vscode#546

* fix docs

* fix quickstart

* add types
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
native-server Related to the native server notebook Related to (Jupyter) notebooks
Projects
None yet
Development

No branches or pull requests

7 participants