Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support of symbolic links as entrypoint of function. #1518

Merged
merged 1 commit into from
Oct 16, 2024

Conversation

ohtanim
Copy link
Contributor

@ohtanim ohtanim commented Oct 16, 2024

Summary

If the entrypoint specified in Function is not a regular file but a symbolic link to a python file, the upload to the cluster will be incomplete and execution of the entrypoint will fail. This PR provides support to run with symbolic links/hardlinks in addition to real file.

Details and comments

Creates source_files directory and put the following 2 files. One is real file, another is symbolic linked file.

source_files/
    pattern_with_arguments.py@ -> ../basic/source_files/pattern_with_arguments.py

And then, creates an instance of QiskitFunction with pattern_with_parallel_workflow.py as entrypoint.

function = QiskitFunction(
    title="test",
    entrypoint="pattern_with_parallel_workflow.py",
    working_dir="./source_files/",
)

Above invocation and subsequent upload() are succeeded, but the following error is occurred when running this function.

2024-10-11 12:48:18,794	INFO job_manager.py:531 -- Runtime env is setting up.
python: can't open file '/tmp/ray/session_2024-10-11_12-05-19_925458_1/runtime_resources/working_dir_files/_ray_pkg_e8408231f4c60cc4/pattern_with_parallel_workflow.py': [Errno 2] No such file or directory

This is because of this implementation.

Symbolic link target(=content) will not be included in the archive file. As result, it will not be available in the cluster side.
TarFile provides the solution for this case. The content of the target files can be added to the archive if dereference=True.



with tarfile.open(artifact_file_path, "w", dereference=True) as tar:

This PR will change above 1 line to support both regular file and symbolic/hard links.

@CLAassistant
Copy link

CLAassistant commented Oct 16, 2024

CLA assistant check
All committers have signed the CLA.

@akihikokuroda akihikokuroda self-requested a review October 16, 2024 17:02
Copy link
Collaborator

@akihikokuroda akihikokuroda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@akihikokuroda
Copy link
Collaborator

@ohtanim Thanks for your contributions! It's a good catch. We probably need to add a test for it. @Tansito

Copy link
Collaborator

@psschwei psschwei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch!

@psschwei psschwei merged commit f847612 into Qiskit:main Oct 16, 2024
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants