Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implements Python import statement parser to avoid ast module. #2685

Merged
merged 1 commit into from
Sep 18, 2024

Conversation

dom96
Copy link
Collaborator

@dom96 dom96 commented Sep 10, 2024

Implements a simple parser which looks for import <module> and from <module> import <symbol> statements in Python code. Transitions the package memory snapshot generation to use this new parser which resolves issues with the snapshot leaking user code (previous code used Python's ast module to parse the files which meant they were loaded into memory and had a chance of being captured by the snapshot).

Tested upstream and using the new pyodide-test unit tests.

@dom96 dom96 force-pushed the dominik/EW-8677 branch 9 times, most recently from 0c27713 to dab3faa Compare September 13, 2024 17:07
@dom96 dom96 marked this pull request as ready for review September 13, 2024 17:07
@dom96 dom96 requested review from a team as code owners September 13, 2024 17:07
src/pyodide/internal/snapshot.ts Outdated Show resolved Hide resolved
src/workerd/api/pyodide/pyodide-test.c++ Show resolved Hide resolved
src/workerd/api/pyodide/pyodide-test.c++ Show resolved Hide resolved
src/workerd/api/pyodide/pyodide.c++ Show resolved Hide resolved
src/workerd/api/pyodide/pyodide.c++ Show resolved Hide resolved
src/workerd/api/pyodide/pyodide.c++ Show resolved Hide resolved
src/workerd/api/pyodide/pyodide.c++ Show resolved Hide resolved
src/workerd/api/pyodide/pyodide.c++ Show resolved Hide resolved
src/workerd/api/pyodide/pyodide.c++ Show resolved Hide resolved
}
}

// XXX: jsg doesn't support kj::Vector return types, so this seems to be the only way to do this.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] What's XXX?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dom96 dom96 force-pushed the dominik/EW-8677 branch 3 times, most recently from f05253e to eb48eae Compare September 17, 2024 16:06
@dom96 dom96 merged commit 31b17c8 into main Sep 18, 2024
12 of 13 checks passed
@dom96 dom96 deleted the dominik/EW-8677 branch September 18, 2024 09:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants