Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement container_pid_to_host_pid() #955

Merged
merged 9 commits into from
Jun 10, 2023

Conversation

kyujin-cho
Copy link
Member

This PR implements ai.backend.agent.utils.container_pid_to_host_pid() function, which has been left as TODO.

@kyujin-cho kyujin-cho added comp:agent Related to Agent component type:enhance Enhance component, behavior, internals without user-facing features labels Jan 5, 2023
@kyujin-cho kyujin-cho added this to the 23.03 milestone Jan 5, 2023
src/ai/backend/agent/utils.py Outdated Show resolved Hide resolved
src/ai/backend/agent/utils.py Outdated Show resolved Hide resolved
@sanxiyn
Copy link
Contributor

sanxiyn commented Feb 27, 2023

This PR needs an update for cgroup v2 support(#865).

@Yaminyam Yaminyam added the size:L 100~500 LoC label Apr 13, 2023
@github-actions github-actions bot added the comp:manager Related to Manager component label May 28, 2023
Copy link
Member

@achimnol achimnol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a good refactoring.

Though, get_conatiner_process_table() could be reimplemented using /proc/{pid}/status's NSPid field (ref: https://man7.org/linux/man-pages/man5/proc.5.html) following get_host_process_table(), as long as we're on the Linux kernel 4.1 or later, like the existing NSpid-based code path for converting individual pids. (A notable exception is CentOS 7... and we may keep the current implementation as a fallback for CentOS 7 users and macOS development setups.)

It would be better to avoid spawning subprocesses for this kind of simple queries whenever possible. Even the mapping query itself could be rewritten to just read a single line of /proc/{pid}/status without contacting the Docker daemon API at all. I think reading the whole container process table could be also using the procfs NSpid field.

I think we could rewrite get_container_process_table() as a follow-up PR.

@achimnol achimnol enabled auto-merge June 10, 2023 06:06
@achimnol achimnol added this pull request to the merge queue Jun 10, 2023
Merged via the queue into main with commit 96373fb Jun 10, 2023
@achimnol achimnol deleted the feature/container-pid-to-host-pid branch June 10, 2023 06:10
achimnol pushed a commit that referenced this pull request Jun 10, 2023
backported-from: main
Backported-to: 23.03
@achimnol achimnol mentioned this pull request Dec 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp:agent Related to Agent component comp:manager Related to Manager component size:L 100~500 LoC type:enhance Enhance component, behavior, internals without user-facing features
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants