Use jupyer-kernel-gateway for ipython executor #1748

jackgerrits · 2024-02-21T16:00:30Z

Why are these changes needed?

This uses the jupyter-kernel-gateway to manage kernels for the ipython based executor. This is desirable as it allows us to have a common interface to interact with kernels irrespective of where they are executing. In this PR I migrate the embedded executor. I will follow this up with a docker container based version.

Related issue number

#1421

Checks

I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
I've added tests (if relevant) corresponding to the changes introduced in this PR.
I've made sure all auto checks have passed.

test/coding/test_embedded_ipython_code_executor.py

codecov-commenter · 2024-02-21T16:05:00Z

Codecov Report

Attention: Patch coverage is 13.86139% with 261 lines in your changes are missing coverage. Please review.

Project coverage is 47.95%. Comparing base (ac15996) to head (4543624).

Files	Patch %	Lines
autogen/coding/jupyter/jupyter_client.py	10.09%	95 Missing and 3 partials ⚠️
autogen/coding/jupyter_code_executor.py	12.37%	85 Missing ⚠️
autogen/coding/jupyter/local_jupyter_server.py	0.00%	69 Missing and 1 partial ⚠️
autogen/coding/factory.py	0.00%	2 Missing and 1 partial ⚠️
autogen/coding/embedded_ipython_code_executor.py	60.00%	2 Missing ⚠️
autogen/coding/jupyter/__init__.py	50.00%	2 Missing ⚠️
autogen/coding/jupyter/base.py	92.30%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1748      +/-   ##
==========================================
+ Coverage   39.13%   47.95%   +8.82%     
==========================================
  Files          57       62       +5     
  Lines        6174     6473     +299     
  Branches     1384     1554     +170     
==========================================
+ Hits         2416     3104     +688     
+ Misses       3564     3118     -446     
- Partials      194      251      +57

Flag	Coverage Δ
unittests	`47.87% <13.86%> (+8.74%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

jackgerrits · 2024-02-21T16:06:59Z

Most of these change should be covered by the existing tests - but I don't think they are running since the dependencies are missing on the build machines

autogen/coding/embedded_ipython_code_executor.py

* checkpoint async based * Implement jupyter client and use jupyer gateway * update deps * address comments * add missing parenthesis * Update build.yml * CI fixes * change requirement name * debug * print stderr * dont seek * show token * mitigaton for windows bug * use hex token to avoid - in token * formatting * put back in place original while the windows bug exists * lint * Update autogen/coding/jupyter_code_executor.py * Update jupyter_code_executor.py * Update test_embedded_ipython_code_executor.py * Update setup.py * Update build.yml * fix nameerror --------- Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>

jackgerrits added 2 commits February 21, 2024 09:00

checkpoint async based

9248983

Implement jupyter client and use jupyer gateway

bac866b

jackgerrits had a problem deploying to openai1 February 21, 2024 16:00 — with GitHub Actions Failure

jackgerrits commented Feb 21, 2024

View reviewed changes

test/coding/test_embedded_ipython_code_executor.py Show resolved Hide resolved

update deps

da0fc4e

jackgerrits had a problem deploying to openai1 February 21, 2024 16:02 — with GitHub Actions Failure

jackgerrits commented Feb 21, 2024

View reviewed changes

autogen/coding/embedded_ipython_code_executor.py Outdated Show resolved Hide resolved

ekzhu self-requested a review February 21, 2024 17:33

jackgerrits had a problem deploying to openai1 February 23, 2024 18:26 — with GitHub Actions Failure

Merge branch 'main' into jupyter_client

4543624

jackgerrits had a problem deploying to openai1 February 23, 2024 18:28 — with GitHub Actions Failure

jackgerrits temporarily deployed to openai1 February 23, 2024 18:28 — with GitHub Actions Inactive

jackgerrits had a problem deploying to openai1 February 23, 2024 18:28 — with GitHub Actions Failure

jackgerrits temporarily deployed to openai1 February 23, 2024 18:28 — with GitHub Actions Inactive

jackgerrits had a problem deploying to openai1 February 23, 2024 18:28 — with GitHub Actions Failure

jackgerrits temporarily deployed to openai1 February 23, 2024 18:28 — with GitHub Actions Inactive

jackgerrits had a problem deploying to openai1 February 23, 2024 18:28 — with GitHub Actions Failure

jackgerrits had a problem deploying to openai1 February 23, 2024 18:57 — with GitHub Actions Failure

jackgerrits temporarily deployed to openai1 February 23, 2024 18:57 — with GitHub Actions Inactive

sonichi enabled auto-merge February 23, 2024 19:06

sonichi added this pull request to the merge queue Feb 23, 2024

Merged via the queue into microsoft:main with commit fbc2f6e Feb 23, 2024
49 of 57 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use jupyer-kernel-gateway for ipython executor #1748

Use jupyer-kernel-gateway for ipython executor #1748

jackgerrits commented Feb 21, 2024

codecov-commenter commented Feb 21, 2024 •

edited

Loading

jackgerrits commented Feb 21, 2024

Use jupyer-kernel-gateway for ipython executor #1748

Use jupyer-kernel-gateway for ipython executor #1748

Conversation

jackgerrits commented Feb 21, 2024

Why are these changes needed?

Related issue number

Checks

codecov-commenter commented Feb 21, 2024 • edited Loading

Codecov Report

jackgerrits commented Feb 21, 2024

codecov-commenter commented Feb 21, 2024 •

edited

Loading