Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow listdir in /proc in the event loop #120969

Closed
wants to merge 1 commit into from
Closed

Conversation

knyar
Copy link
Contributor

@knyar knyar commented Jul 1, 2024

Proposed change

It seems like there's some automation now that detects blocking IO in Home Assistant, and I've been looking at #120778 which reported an os.listdir() call made by the metric library that the prometheus integration uses.

Taking a closer look in block_async_io, it seems like some operations in /proc have been deemed acceptable, so perhaps the right change here is to extend that exception list to os.listdir?

Please let me know if I misunderstood the intent here - I am happy to move the blocking code to be ran by the executor as suggested in developer docs.

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Deprecation (breaking change to happen in the future)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • I have followed the perfect PR recommendations
  • The code has been formatted using Ruff (ruff format homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
  • Untested files have been added to .coveragerc.

To help with the load of incoming pull requests:

@knyar knyar requested a review from a team as a code owner July 1, 2024 22:31
Copy link

@home-assistant home-assistant bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @knyar,

When attempting to inspect the commits of your pull request for CLA signature status among all authors we encountered commit(s) which were not linked to a GitHub account, thus not allowing us to determine their status(es).

The commits that are missing a linked GitHub account are the following:

Unfortunately, we are unable to accept this pull request until this situation is corrected.

Here are your options:

  1. If you had an email address set for the commit that simply wasn't linked to your GitHub account you can link that email now and it will retroactively apply to your commits. The simplest way to do this is to click the link to one of the above commits and look for a blue question mark in a blue circle in the top left. Hovering over that bubble will show you what email address you used. Clicking on that button will take you to your email address settings on GitHub. Just add the email address on that page and you're all set. GitHub has more information about this option in their help center.

  2. If you didn't use an email address at all, it was an invalid email, or it's one you can't link to your GitHub, you will need to change the authorship information of the commit and your global Git settings so this doesn't happen again going forward. GitHub provides some great instructions on how to change your authorship information in their help center.

    • If you only made a single commit you should be able to run
      git commit --amend --author="Author Name <email@address.com>"
      
      (substituting "Author Name" and "email@address.com" for your actual information) to set the authorship information.
    • If you made more than one commit and the commit with the missing authorship information is not the most recent one you have two options:
      1. You can re-create all commits missing authorship information. This is going to be the easiest solution for developers that aren't extremely confident in their Git and command line skills.
      2. You can use this script that GitHub provides to rewrite history. Please note: this should be used only if you are very confident in your abilities and understand its impacts.
    • Whichever method you choose, I will come by to re-check the pull request once you push the fixes to this branch.

We apologize for this inconvenience, especially since it usually bites new contributors to Home Assistant. We hope you understand the need for us to protect ourselves and the great community we all have built legally. The best thing to come out of this is that you only need to fix this once and it benefits the entire Home Assistant and GitHub community.

Thanks, I look forward to checking this PR again soon! ❤️

@home-assistant
Copy link

home-assistant bot commented Jul 1, 2024

Please take a look at the requested changes, and use the Ready for review button when you are done, thanks 👍

Learn more about our pull request process.

@home-assistant home-assistant bot marked this pull request as draft July 1, 2024 22:31
@knyar knyar marked this pull request as ready for review July 1, 2024 22:57
@knyar
Copy link
Contributor Author

knyar commented Jul 1, 2024

cc @bdraco

@bdraco
Copy link
Member

bdraco commented Jul 1, 2024

I’m not sure if listdir on /proc/self/fd can block or not. We need to look at the kernel sources to see if it’s safe to exclude or not

@bdraco
Copy link
Member

bdraco commented Jul 2, 2024

Digging though https://elixir.bootlin.com/linux/v5.19.4/source/fs/proc, it does look like listing /proc/self/fd can block as there are quite a few places it can lock so that lookup should run in the executor

The other use cases we allow are are mainly for reading cpu/memory stats which doesn't appear to block https://elixir.bootlin.com/linux/v5.19.4/source/fs/proc/stat.c. That exclude should be more narrow but that's another issue

@knyar
Copy link
Contributor Author

knyar commented Jul 3, 2024

Thank you for the quick response here! Closing this in favor of #121058.

@knyar knyar closed this Jul 3, 2024
@github-actions github-actions bot locked and limited conversation to collaborators Jul 4, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
2 participants