Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jupyter kernel on vscode gives error about variables declared on above cells #932

Open
ceifa opened this issue Sep 21, 2023 · 61 comments
Open
Assignees
Labels
jupyter Also see https://github.com/denoland/deno/issues?q=is%3Aopen+label%3Ajupyter+label%3Alsp

Comments

@ceifa
Copy link

ceifa commented Sep 21, 2023

To Reproduce

  1. Open deno kernel on vscode
  2. Create a variable in the first cell
  3. Use it on the second one

Expected behavior

None errors

Screenshots

image

Versions

vscode: 1.82.2 deno: 1.37 extension: v3.23.1

@bartlomieju

@bartlomieju
Copy link
Member

@nayeemrmn could you take a look what's going on here?

@amunger
Copy link

amunger commented Sep 29, 2023

This will happen in the deno language server isn't being used - try adding "deno.enable": true to your settings and see if the error goes away.

It looks like you can also put your notebooks in a specific directory and add that directory to deno.enablePaths as recommended in the setting description.

@amunger
Copy link

amunger commented Sep 29, 2023

But I would say it should be safe for the extension to always use the deno language server for a notebook that has a deno kernel selected.

@ceifa
Copy link
Author

ceifa commented Sep 29, 2023

But I would say it should be safe for the extension to always use the deno language server for a notebook that has a deno kernel selected.

I agree. Specifically because I don't want deno to be enabled on the global settings and I also don't want to save the notebook file sometimes.

@ghing
Copy link

ghing commented Dec 5, 2023

I'm also seeing this issue and have "deno.enable": true in my .vscode/settings.json.

@scarf005
Copy link

i'm wondering how this could be solved implementation-wise. my naive first impression was:

  1. join all typescript cells
  2. send the joined cell into tsc
  3. parse tsc diagnostics and re-map line numbers into splitted cells

@nayeemrmn
Copy link
Collaborator

I forgot to give an update here, this is blocked by gluon-lang/lsp-types#268 which implements this API in the protocol https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#notebookDocument_synchronization. This provides the minimum of getting the accurate order of the cells etc.

After that it's still going to be difficult and we'd likely have to do what @scarf005 proposed above. On the surface this isn't a problem with VSCode's built-in node TS server because it has notebook cells in script mode, where all vars are global. We might alternatively end up doing something similar.

@bartlomieju
Copy link
Member

Would this PR help somehow? denoland/deno#21518

@pekam
Copy link

pekam commented Jan 8, 2024

I'm also seeing this issue and have "deno.enable": true in my .vscode/settings.json.

Same here. The Deno APIs are enabled as shown by this screenshot:
image

Jupyter support is the only reason why I started using Deno, and this issue is a blocker for me.

@Hexodus
Copy link

Hexodus commented Feb 10, 2024

What a pitty, I was so excited to try out Deno inside of jupyter notebooks but having variables that don't remember their values makes it a no go. And yes, Deno is enabled in this workspace.
image

@sigmaSd
Copy link

sigmaSd commented Feb 10, 2024

Just as a reminder, the code does work as expected, this issue is just about the lsp giving wrong diagnostics, but if you run the cell it will work.

@vanb
Copy link

vanb commented Mar 2, 2024

Just as a reminder, the code does work as expected, this issue is just about the lsp giving wrong diagnostics, but if you run the cell it will work.

Unfortunately, even if it does execute, it's frustrating to lose all type information and be downgraded to any after the first cell. So, no code completion or type checking, effectively losing all benefits of typescript. Any ETA on when this might be fixed?

@dnszero
Copy link

dnszero commented Apr 15, 2024

Anyone have a work-around for this?

As it stands, it's nearly unusable. :(

@matijagrcic
Copy link

This would be great to have.

I've read the comments:

#932 (comment)

I forgot to give an update here, this is blocked by gluon-lang/lsp-types#268 which implements this API in the protocol https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#notebookDocument_synchronization. This provides the minimum of getting the accurate order of the cells etc.

After that it's still going to be difficult and we'd likely have to do what @scarf005 proposed above. On the surface this isn't a problem with VSCode's built-in node TS server because it has notebook cells in script mode, where all vars are global. We might alternatively end up doing something similar.

denoland/deno#21518 (comment)

I am no longer working on this PR.

If I remember correctly, the Deno language server does not have proper notebook support but treats every cell like a separate document. This is the default fallback mechanism for notebooks if the language server does not support notebooks. That behavior causes some problems; for example, the type of variables from the previous cell is unknown in the subsequent cells. Fixing this would require the language server to treat all cells as a single file, but that would come with other issues, as the cells are not necessarily executed from top to bottom. However, that would still be more useful than the current implementation. Afaik, no language server for a statically typed language currently has good support for notebooks.

The language server protocol supports notebooks, but we need tower-lsp to do the same. I opened a PR over there (ebkalderon/tower-lsp#401), but that is blocked by gluon-lang/lsp-types#266, which is blocked by gluon-lang/lsp-types#266.

I am closing this PR because I have given up on using deno for notebooks.

So wondering can import_maps be used to propagate type information across the cells?

So far i'm leveraging deps.ts and it's not ideal as it still complains about unused values etc, but at least i'm getting the intelliSense.

The standard practice for solving this problem in Deno is to create a deps.ts file. All required remote dependencies are referenced in this file and the required methods and classes are re-exported. The dependent local modules then reference the deps.ts rather than the remote dependencies.

@ple1n
Copy link

ple1n commented May 17, 2024

@nayeemrmn gluon PR has been merged can this be resumed

gluon-lang/lsp-types#282

@ple1n
Copy link

ple1n commented May 17, 2024

on top of that it'd be nice to have chrome-devtools-like object inspector (that expands as you click on the items) in notebook cell output

@matijagrcic
Copy link

on top of that it'd be nice to have chrome-devtools-like object inspector (that expands as you click on the items) in notebook cell output

You can already use rich display for that if you want https://deno.land/x/display@v1.1.2 and build custom formatters.

@jarrodcolburn
Copy link

jarrodcolburn commented Aug 11, 2024

The irony is that because the jupyter extension works fine for typescript by itself and won't warn of variables, and gives types hints.,
When the deno extension is disabled (jupyter is enabled) it works fine.

So my temporary workaround is...

  • Create directory in project root to store .ipbyn files (example: ./notebook)
  • disable deno extension for that directory in workspace's settings.json example:
{
    "deno.enable":true, 
    "deno.disablePaths": [
        "./notebooks"
    ]
}
  • Only rely on jupyter extension

image

@redking00
Copy link

redking00 commented Nov 4, 2024

@ple1n Can you try with "deno.path":"/home/phy/.cargo/bin/deno" ? Thanks

@ple1n
Copy link

ple1n commented Nov 4, 2024

image

I created a new vscode profile, and set deno.path

@ple1n
Copy link

ple1n commented Nov 4, 2024

why does deno exit. it crashed? and what is that object serialize it and print it ig.

@ple1n
Copy link

ple1n commented Nov 4, 2024

image

deno is compiled from source

@redking00
Copy link

@ple1n
Seems deno is exiting. The object is the spawned process. My guess is it's a problem opening the zeromq sockets. Can you try this?

Save this as connection_file
{ "ip": "127.0.0.1", "iopub_port": 40885, "shell_port": 40886, "control_port": 40887, "hb_port": 40888, "stdin_port": 40889, "transport": "tcp", "signature_scheme": "sha256", "key": "ba497f9f-1606-4dbd-aa45-cfc8a70ffc57" }

Then launch
deno jupyter --kernel --conn connection_file

Is deno exiting after that?

@ple1n
Copy link

ple1n commented Nov 4, 2024

I cloned your repo and set it up. seems source maps are not working cant debug it.

image

btw is this gonna do anything js console is better for objects. its not showing up in vs dev tools

@ple1n
Copy link

ple1n commented Nov 4, 2024

image

@redking00

@ple1n
Copy link

ple1n commented Nov 4, 2024

➜  ~ deno jupyter --kernel --conn connfile 
Warning "deno jupyter" is unstable and might change in the future.
error: Failed to acquire startup data

@ple1n
Copy link

ple1n commented Nov 4, 2024

image

@ple1n
Copy link

ple1n commented Nov 4, 2024

I killed that process. It seems to be some left-over process that occupied the ports. And it worked

@redking00
Copy link

I'm glad to hear that! @ple1n Thanks for your interest

@ple1n
Copy link

ple1n commented Nov 4, 2024

image

@ple1n
Copy link

ple1n commented Nov 4, 2024

I mean how should the auto completion for that object be done. seems like a non trivial issue. infer the typing based on the object and send it back to lsp or such? and is that within the scope of your fork/pr / requiring deno-team intervention

@redking00
Copy link

I mean how should the auto completion for that object be done. seems like a non trivial issue. infer the typing based on the object and send it back to lsp or such? and is that within the scope of your fork/pr / requiring deno-team intervention

I have no idea how something like that could be done. It certainly wasn't on my to-do list for this project, but I'm open to helping in any way I can.

@ple1n
Copy link

ple1n commented Nov 4, 2024

It conflicts with other instances of the extension, ie I cant run multiple notebooks @redking00

@redking00
Copy link

@ple1n There is no problem running multiple notebooks on the same VSCode instance so I assume you mean multiple VSCode instances. It's probably the same problem with the ports. I have to add a random initial offset.

@redking00
Copy link

@ple1n Just updated to v0.0.5. Can you check it?. I tested two VSCode instances with no problem.

@ple1n
Copy link

ple1n commented Nov 4, 2024

@ple1n There is no problem running multiple notebooks on the same VSCode instance so I assume you mean multiple VSCode instances. It's probably the same problem with the ports. I have to add a random initial offset.

Yes exactly

image

it pops up the error every second I write the sql statement (obviosuly it was kinda sending the sql to LSP for some reason). I cant suppress the notification it gets annoying

@ple1n
Copy link

ple1n commented Nov 4, 2024

no. it errors elsewhere too, pops up every key-hit

@ple1n
Copy link

ple1n commented Nov 4, 2024


[Trace - 5:10:19 AM] Received response 'textDocument/codeAction - (5017)' in 8ms. Request failed: Invalid request (-32600).
[Error - 5:10:19 AM] Request textDocument/codeAction failed.
  Message: Invalid request
  Code: -32600 
[Trace - 5:10:19 AM] Sending request 'textDocument/codeAction - (5018)'.
Failed to request to tsserver tsc error: TypeError: Cannot read properties of undefined (reading 'kind')
    at isVariableDeclaration (ext:deno_tsc/00_typescript.js:30518:15)
    at isSingleVariableDeclaration (ext:deno_tsc/00_typescript.js:146058:10)
    at tryGetFunctionFromVariableDeclaration (ext:deno_tsc/00_typescript.js:146061:8)
    at getFunctionInfo (ext:deno_tsc/00_typescript.js:146046:16)
    at Object.getRefactorActionsToConvertFunctionExpressions [as getAvailableActions] (ext:deno_tsc/00_typescript.js:145971:16)
    at ext:deno_tsc/00_typescript.js:143671:222
    at flatMapIterator (ext:deno_tsc/00_typescript.js:2545:19)
    at flatMapIterator.next (<anonymous>)
    at arrayFrom (ext:deno_tsc/00_typescript.js:3038:14)
    at Object.getApplicableRefactors (ext:deno_tsc/00_typescript.js:143669:10)
For request: [3193, "getApplicableRefactors", ["file:///home/phy/drugbankdb/peek1.nb.ts",{"pos":953,"end":953},{"quotePreference":"double","includeCompletionsForModuleExports":true,"includeCompletionsForImportStatements":true,"includeCompletionsWithSnippetText":true,"includeAutomaticOptionalChainCompletions":true,"includeCompletionsWithInsertText":true,"includeCompletionsWithClassMemberSnippets":true,"includeCompletionsWithObjectLiteralMethodSnippets":true,"useLabelDetailsInCompletionEntries":true,"allowIncompleteCompletions":true,"importModuleSpecifierPreference":"shortest","importModuleSpecifierEnding":"index","allowTextChangesInNewFiles":true,"providePrefixAndSuffixTextForRename":true,"provideRefactorNotApplicableReason":true,"jsxAttributeCompletionStyle":"auto","includeInlayParameterNameHints":"none","includeInlayParameterNameHintsWhenArgumentMatchesName":false,"includeInlayFunctionParameterTypeHints":false,"includeInlayVariableTypeHints":false,"includeInlayVariableTypeHintsWhenTypeMatchesName":false,"includeInlayPropertyDeclarationTypeHints":false,"includeInlayFunctionLikeReturnTypeHints":false,"includeInlayEnumMemberValueHints":false,"autoImportFileExcludePatterns":[],"interactiveInlayHints":true,"preferTypeOnlyAutoImports":false},"implicit",""], "file:///home/phy/drugbankdb/", null]
[Trace - 5:10:19 AM] Received response 'textDocument/codeAction - (5018)' in 6ms. Request failed: Invalid request (-32600).
[Error - 5:10:19 AM] Request textDocument/codeAction failed.
  Message: Invalid request
  Code: -32600 
[Trace - 5:10:20 AM] Sending request 'textDocument/inlayHint - (5019)'.
[Trace - 5:10:20 AM] Received response 'textDocument/inlayHint - (5019)' in 13ms.
[Trace - 5:10:40 AM] Sending request 'textDocument/codeLens - (5020)'.
[Trace - 5:10:40 AM] Received response 'textDocument/codeLens - (5020)' in 6ms.
[Trace - 5:10:42 AM] Sending request 'textDocument/codeAction - (5021)'.
Failed to request to tsserver tsc error: TypeError: Cannot read properties of undefined (reading 'kind')
    at isVariableDeclaration (ext:deno_tsc/00_typescript.js:30518:15)
    at isSingleVariableDeclaration (ext:deno_tsc/00_typescript.js:146058:10)
    at tryGetFunctionFromVariableDeclaration (ext:deno_tsc/00_typescript.js:146061:8)
    at getFunctionInfo (ext:deno_tsc/00_typescript.js:146046:16)
    at Object.getRefactorActionsToConvertFunctionExpressions [as getAvailableActions] (ext:deno_tsc/00_typescript.js:145971:16)
    at ext:deno_tsc/00_typescript.js:143671:222
    at flatMapIterator (ext:deno_tsc/00_typescript.js:2545:19)
    at flatMapIterator.next (<anonymous>)
    at arrayFrom (ext:deno_tsc/00_typescript.js:3038:14)
    at Object.getApplicableRefactors (ext:deno_tsc/00_typescript.js:143669:10)
For request: [3194, "getApplicableRefactors", ["file:///home/phy/drugbankdb/peek1.nb.ts",{"pos":1027,"end":1027},{"quotePreference":"double","includeCompletionsForModuleExports":true,"includeCompletionsForImportStatements":true,"includeCompletionsWithSnippetText":true,"includeAutomaticOptionalChainCompletions":true,"includeCompletionsWithInsertText":true,"includeCompletionsWithClassMemberSnippets":true,"includeCompletionsWithObjectLiteralMethodSnippets":true,"useLabelDetailsInCompletionEntries":true,"allowIncompleteCompletions":true,"importModuleSpecifierPreference":"shortest","importModuleSpecifierEnding":"index","allowTextChangesInNewFiles":true,"providePrefixAndSuffixTextForRename":true,"provideRefactorNotApplicableReason":true,"jsxAttributeCompletionStyle":"auto","includeInlayParameterNameHints":"none","includeInlayParameterNameHintsWhenArgumentMatchesName":false,"includeInlayFunctionParameterTypeHints":false,"includeInlayVariableTypeHints":false,"includeInlayVariableTypeHintsWhenTypeMatchesName":false,"includeInlayPropertyDeclarationTypeHints":false,"includeInlayFunctionLikeReturnTypeHints":false,"includeInlayEnumMemberValueHints":false,"autoImportFileExcludePatterns":[],"interactiveInlayHints":true,"preferTypeOnlyAutoImports":false},"implicit",""], "file:///home/phy/drugbankdb/", null]
[Trace - 5:10:42 AM] Received response 'textDocument/codeAction - (5021)' in 8ms. Request failed: Invalid request (-32600).
[Error - 5:10:42 AM] Request textDocument/codeAction failed.
  Message: Invalid request
  Code: -32600 
[Trace - 5:10:42 AM] Sending request 'textDocument/inlayHint - (5022)'.
[Trace - 5:10:42 AM] Received response 'textDocument/inlayHint - (5022)' in 5ms.
[Trace - 5:10:42 AM] Sending request 'textDocument/foldingRange - (5023)'.
[Trace - 5:10:42 AM] Received response 'textDocument/foldingRange - (5023)' in 6ms.
[Trace - 5:10:42 AM] Sending request 'textDocument/inlayHint - (5024)'.
[Trace - 5:10:42 AM] Sending request 'textDocument/codeAction - (5025)'.
[Trace - 5:10:42 AM] Received response 'textDocument/inlayHint - (5024)' in 5ms.
Failed to request to tsserver tsc error: TypeError: Cannot read properties of undefined (reading 'kind')
    at isVariableDeclaration (ext:deno_tsc/00_typescript.js:30518:15)
    at isSingleVariableDeclaration (ext:deno_tsc/00_typescript.js:146058:10)
    at tryGetFunctionFromVariableDeclaration (ext:deno_tsc/00_typescript.js:146061:8)
    at getFunctionInfo (ext:deno_tsc/00_typescript.js:146046:16)
    at Object.getRefactorActionsToConvertFunctionExpressions [as getAvailableActions] (ext:deno_tsc/00_typescript.js:145971:16)
    at ext:deno_tsc/00_typescript.js:143671:222
    at flatMapIterator (ext:deno_tsc/00_typescript.js:2545:19)
    at flatMapIterator.next (<anonymous>)
    at arrayFrom (ext:deno_tsc/00_typescript.js:3038:14)
    at Object.getApplicableRefactors (ext:deno_tsc/00_typescript.js:143669:10)
For request: [3196, "getApplicableRefactors", ["file:///home/phy/drugbankdb/peek1.nb.ts",{"pos":1035,"end":1035},{"quotePreference":"double","includeCompletionsForModuleExports":true,"includeCompletionsForImportStatements":true,"includeCompletionsWithSnippetText":true,"includeAutomaticOptionalChainCompletions":true,"includeCompletionsWithInsertText":true,"includeCompletionsWithClassMemberSnippets":true,"includeCompletionsWithObjectLiteralMethodSnippets":true,"useLabelDetailsInCompletionEntries":true,"allowIncompleteCompletions":true,"importModuleSpecifierPreference":"shortest","importModuleSpecifierEnding":"index","allowTextChangesInNewFiles":true,"providePrefixAndSuffixTextForRename":true,"provideRefactorNotApplicableReason":true,"jsxAttributeCompletionStyle":"auto","includeInlayParameterNameHints":"none","includeInlayParameterNameHintsWhenArgumentMatchesName":false,"includeInlayFunctionParameterTypeHints":false,"includeInlayVariableTypeHints":false,"includeInlayVariableTypeHintsWhenTypeMatchesName":false,"includeInlayPropertyDeclarationTypeHints":false,"includeInlayFunctionLikeReturnTypeHints":false,"includeInlayEnumMemberValueHints":false,"autoImportFileExcludePatterns":[],"interactiveInlayHints":true,"preferTypeOnlyAutoImports":false},"implicit",""], "file:///home/phy/drugbankdb/", null]
[Trace - 5:10:42 AM] Sending request 'textDocument/codeLens - (5026)'.
[Trace - 5:10:42 AM] Received response 'textDocument/codeAction - (5025)' in 7ms. Request failed: Invalid request (-32600).
[Error - 5:10:42 AM] Request textDocument/codeAction failed.
  Message: Invalid request
  Code: -32600 
[Trace - 5:10:42 AM] Received response 'textDocument/codeLens - (5026)' in 4ms.

@ple1n
Copy link

ple1n commented Nov 4, 2024

and as predicted, by intuition. it disappears after reloading window

@redking00
Copy link

@ple1n Please open an issue and explain in detail how I can replicate your problem.

@redking00
Copy link

redking00 commented Nov 14, 2024

For those interested in a quick test on the lsp thing... a codespaces template.

@nayeemrmn , @bartlomieju, Any interest in this or do I keep the fork? Maybe it can be included as an opt-in setting for notebooks users until the real lsp is ready.

@ple1n
Copy link

ple1n commented Nov 15, 2024

check my project out https://github.com/planetoryd/drugbankdb

I've been using this notebook feature with SurrealDB for data exploration. Its publicly available drugbank data you just need an account to download.

I am not associated with anyone else its just for personal needs.

The alternative is to use ELK stack, which does not work with graph data yet. Or use MongoDB stack, which did not work out for me, or has good features paywalled.

And I am NOT gonna use python+pylance for surrealdb. It will suck.

@ple1n
Copy link

ple1n commented Nov 15, 2024

image

@bartlomieju
Copy link
Member

For those interested in a quick test on the lsp thing... a codespaces template.

@nayeemrmn , @bartlomieju, Any interest in this or do I keep the fork? Maybe it can be included as an opt-in setting for notebooks users until the real lsp is ready.

Thanks @redking00, @nayeemrmn is probably 50% done with the LSP integration at this point, but we have a few more pressing issues to fix before then. I'm not super keen on bringing a lot of new functionality into the extension as we try to keep it slim and offload most of the work to the LSP so it's easily testable. So I'd say it might be better to keep the fork going for now.

@redking00
Copy link

@bartlomieju, @nayeemrmn,
I love knowing that work is being done on the lsp.
Being able to run the kernel without python seems to me to be something you should take into account for the future.
And the serializer... I think it is very useful for maintaining dependencies between notebooks, especially in the field of literate programming.
In any case, I understand what you are saying.
I will keep the fork.
I love your work! Keep it up!

@bartlomieju
Copy link
Member

@redking00 I might be missing some crucial detail here - maybe you could open issues in the Deno repo with suggestions for the kernel and the serializer?

@redking00
Copy link

@bartlomieju Both are vscode stuff.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
jupyter Also see https://github.com/denoland/deno/issues?q=is%3Aopen+label%3Ajupyter+label%3Alsp
Projects
None yet
Development

No branches or pull requests