Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Salford City Council: 'NoneType' has no attribute 'find_all' #1097

Closed
cschinis opened this issue Dec 21, 2024 · 3 comments · Fixed by #1100
Closed

Salford City Council: 'NoneType' has no attribute 'find_all' #1097

cschinis opened this issue Dec 21, 2024 · 3 comments · Fixed by #1100
Labels
bug Something isn't working

Comments

@cschinis
Copy link

cschinis commented Dec 21, 2024

Name of Council

Salford City Council

Issue Information

Not exactly sure when it stopped working but it worked as expected on 12 December.

Issue on Home Assistant is "Failed setup, will retry: Unexpected error: 'NoneType' has no attribute 'find_all'"
image

Issue also verified/experience by two other Home Assistant user who also live in Salford.

Issue can be very easily replicated in Home Assistant.

UK Bin Collection Integration version: 0.123.2
Home Assistant Integration: 2024.12.5

Debug Log below:
AttributeError: 'NoneType' object has no attribute 'find_all'
2024-12-21 15:33:44.871 ERROR (MainThread) [custom_components.uk_bin_collection] [UKBinCollection] Unexpected error: 'NoneType' object has no attribute 'find_all'
Traceback (most recent call last):
File "/config/custom_components/uk_bin_collection/init.py", line 250, in _async_update_data
data = await asyncio.wait_for(
^^^^^^^^^^^^^^^^^^^^^^^
...<2 lines>...
)
^
File "/usr/local/lib/python3.13/asyncio/tasks.py", line 507, in wait_for
return await fut
^^^^^^^^^
File "/usr/local/lib/python3.13/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.13/site-packages/uk_bin_collection/uk_bin_collection/collect_data.py", line 101, in run
return self.client_code(

council_module.CouncilClass(),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<9 lines>...
council_module_str=self.parsed_args.module,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/usr/local/lib/python3.13/site-packages/uk_bin_collection/uk_bin_collection/collect_data.py", line 121, in client_code
return get_bin_data_class.template_method(address_url, **kwargs)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.13/site-packages/uk_bin_collection/uk_bin_collection/get_bin_data.py", line 61, in template_method
bin_data_dict = self.get_and_parse_data(this_url, **kwargs)
File "/usr/local/lib/python3.13/site-packages/uk_bin_collection/uk_bin_collection/get_bin_data.py", line 82, in get_and_parse_data
bin_data_dict = self.parse_data(page, url=address_url, **kwargs)
File "/usr/local/lib/python3.13/site-packages/uk_bin_collection/uk_bin_collection/councils/SalfordCityCouncil.py", line 42, in parse_data
bin_lists = div_element.find_all("ul")
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'find_all'
2024-12-21 15:34:25.655 ERROR (MainThread) [custom_components.uk_bin_collection] [UKBinCollection] Unexpected error: 'NoneType' object has no attribute 'find_all'
Traceback (most recent call last):
File "/config/custom_components/uk_bin_collection/init.py", line 250, in _async_update_data
data = await asyncio.wait_for(
^^^^^^^^^^^^^^^^^^^^^^^
...<2 lines>...
)
^
File "/usr/local/lib/python3.13/asyncio/tasks.py", line 507, in wait_for
return await fut
^^^^^^^^^
File "/usr/local/lib/python3.13/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.13/site-packages/uk_bin_collection/uk_bin_collection/collect_data.py", line 101, in run
return self.client_code(
~~~~~~~~~~~~~~~~^
council_module.CouncilClass(),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<9 lines>...
council_module_str=self.parsed_args.module,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/usr/local/lib/python3.13/site-packages/uk_bin_collection/uk_bin_collection/collect_data.py", line 121, in client_code
return get_bin_data_class.template_method(address_url, **kwargs)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.13/site-packages/uk_bin_collection/uk_bin_collection/get_bin_data.py", line 61, in template_method
bin_data_dict = self.get_and_parse_data(this_url, **kwargs)
File "/usr/local/lib/python3.13/site-packages/uk_bin_collection/uk_bin_collection/get_bin_data.py", line 82, in get_and_parse_data
bin_data_dict = self.parse_data(page, url=address_url, **kwargs)
File "/usr/local/lib/python3.13/site-packages/uk_bin_collection/uk_bin_collection/councils/SalfordCityCouncil.py", line 42, in parse_data
bin_lists = div_element.find_all("ul")
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'find_all'
2024-12-21 15:35:46.208 ERROR (MainThread) [custom_components.uk_bin_collection] [UKBinCollection] Unexpected error: 'NoneType' object has no attribute 'find_all'
Traceback (most recent call last):
File "/config/custom_components/uk_bin_collection/init.py", line 250, in _async_update_data
data = await asyncio.wait_for(
^^^^^^^^^^^^^^^^^^^^^^^
...<2 lines>...
)
^
File "/usr/local/lib/python3.13/asyncio/tasks.py", line 507, in wait_for
return await fut
^^^^^^^^^
File "/usr/local/lib/python3.13/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.13/site-packages/uk_bin_collection/uk_bin_collection/collect_data.py", line 101, in run
return self.client_code(
~~~~~~~~~~~~~~~~^
council_module.CouncilClass(),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<9 lines>...
council_module_str=self.parsed_args.module,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/usr/local/lib/python3.13/site-packages/uk_bin_collection/uk_bin_collection/collect_data.py", line 121, in client_code
return get_bin_data_class.template_method(address_url, **kwargs)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.13/site-packages/uk_bin_collection/uk_bin_collection/get_bin_data.py", line 61, in template_method
bin_data_dict = self.get_and_parse_data(this_url, **kwargs)
File "/usr/local/lib/python3.13/site-packages/uk_bin_collection/uk_bin_collection/get_bin_data.py", line 82, in get_and_parse_data
bin_data_dict = self.parse_data(page, url=address_url, **kwargs)
File "/usr/local/lib/python3.13/site-packages/uk_bin_collection/uk_bin_collection/councils/SalfordCityCouncil.py", line 42, in parse_data
bin_lists = div_element.find_all("ul")
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'find_all'
2024-12-21 15:35:50.307 WARNING (MainThread) [ring_doorbell.doorbot] Your Ring account does not have an active subscription.
2024-12-21 15:36:19.925 INFO (MainThread) [custom_components.uk_bin_collection] [UKBinCollection] Setting up UK Bin Collection.
2024-12-21 15:36:19.925 DEBUG (MainThread) [custom_components.uk_bin_collection] [UKBinCollection] Retrieved configuration: name=Home, timeout=60, update_interval=12 hours, icon_color_mapping=
2024-12-21 15:36:19.925 DEBUG (MainThread) [custom_components.uk_bin_collection] [UKBinCollection] Built UKBinCollectionApp arguments: ['SalfordCityCouncil', 'https://www.salford.gov.uk/bins-and-recycling/bin-collection-days/your-bin-collections', '--uprn=10004676898']
2024-12-21 15:36:19.925 DEBUG (MainThread) [custom_components.uk_bin_collection] [UKBinCollection] UKBinCollectionApp args: ['SalfordCityCouncil', 'https://www.salford.gov.uk/bins-and-recycling/bin-collection-days/your-bin-collections', '--uprn=10004676898']
2024-12-21 15:36:19.926 DEBUG (MainThread) [custom_components.uk_bin_collection] [UKBinCollection] UKBinCollectionApp initialized and arguments set.
2024-12-21 15:36:19.926 DEBUG (MainThread) [custom_components.uk_bin_collection] [UKBinCollection] DataUpdateCoordinator initialized with update_interval=12:00:00.
2024-12-21 15:36:19.926 DEBUG (MainThread) [custom_components.uk_bin_collection] [UKBinCollection] HouseholdBinCoordinator initialized with update_interval=12 hours.
2024-12-21 15:36:19.927 DEBUG (MainThread) [custom_components.uk_bin_collection] [UKBinCollection] Starting data fetch with timeout=60 seconds.
2024-12-21 15:36:19.927 INFO (MainThread) [custom_components.uk_bin_collection] [UKBinCollection] Fetching latest bin collection data.
2024-12-21 15:36:20.020 ERROR (MainThread) [custom_components.uk_bin_collection] [UKBinCollection] Unexpected error: 'NoneType' object has no attribute 'find_all'
Traceback (most recent call last):
File "/config/custom_components/uk_bin_collection/init.py", line 250, in _async_update_data
data = await asyncio.wait_for(
^^^^^^^^^^^^^^^^^^^^^^^
...<2 lines>...
)
^
File "/usr/local/lib/python3.13/asyncio/tasks.py", line 507, in wait_for
return await fut
^^^^^^^^^
File "/usr/local/lib/python3.13/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.13/site-packages/uk_bin_collection/uk_bin_collection/collect_data.py", line 101, in run
return self.client_code(
~~~~~~~~~~~~~~~~^
council_module.CouncilClass(),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<9 lines>...
council_module_str=self.parsed_args.module,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/usr/local/lib/python3.13/site-packages/uk_bin_collection/uk_bin_collection/collect_data.py", line 121, in client_code
return get_bin_data_class.template_method(address_url, **kwargs)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.13/site-packages/uk_bin_collection/uk_bin_collection/get_bin_data.py", line 61, in template_method
bin_data_dict = self.get_and_parse_data(this_url, **kwargs)
File "/usr/local/lib/python3.13/site-packages/uk_bin_collection/uk_bin_collection/get_bin_data.py", line 82, in get_and_parse_data
bin_data_dict = self.parse_data(page, url=address_url, **kwargs)
File "/usr/local/lib/python3.13/site-packages/uk_bin_collection/uk_bin_collection/councils/SalfordCityCouncil.py", line 42, in parse_data
bin_lists = div_element.find_all("ul")
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'find_all'
2024-12-21 15:36:20.022 DEBUG (MainThread) [custom_components.uk_bin_collection] Finished fetching Home data in 0.095 seconds (success: False)

### Verification

- [X] I searched for similar issues at https://github.com/robbrad/UKBinCollectionData/issues?q=is:issue and found no duplicates
- [X] I have checked my address/postcode/UPRN works on the council's website
- [X] I have provided a detailed explanation of the issue as well as steps to replicate the issue
- [X] I understand that this project is run by volunteer contributors therefore completion of this issue cannot be guaranteed
@cschinis cschinis added the bug Something isn't working label Dec 21, 2024
@cschinis
Copy link
Author

cschinis commented Dec 21, 2024

I've done a little troubleshooting and can report the following:
1.) https://www.salford.gov.uk/bins-and-recycling/bin-collection-days/your-bin-collections/?UPRN=[UPRN redacted] link works as it should if visited directly.

2.) looking at the soup sequence from SalfordCityCouncil.py (lines 38/39), Div class wastefurther is still available on the council website which also looks unchanged to previously.
# Get the div element
div_element = soup.find("div", {"class": "wastefurther"})

However: lines 41/42 seem to fail which gives the error. Specifically find_all
# Get the bins
bin_lists = div_element.find_all("ul")

Using ukbin-ukbc_api_server docker image, the server response is (Also tried with SandwellBoroughCouncil and that works fine)
image

@Smir
Copy link

Smir commented Dec 22, 2024

Same issue for Scarborough, which uses the North Yorkshire api I guess?

Was fine day or so ago

@dp247
Copy link
Collaborator

dp247 commented Dec 22, 2024

Same issue for Scarborough, which uses the North Yorkshire api I guess?

Was fine day or so ago

Please open a new issue and post your HA logs

m26dvd added a commit to m26dvd/UKBinCollectionData that referenced this issue Dec 22, 2024
This was referenced Dec 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants