Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scraper.py - #1816

Open
Bigben83 opened this issue Apr 3, 2024 · 0 comments
Open

Scraper.py - #1816

Bigben83 opened this issue Apr 3, 2024 · 0 comments

Comments

@Bigben83
Copy link

Bigben83 commented Apr 3, 2024

Hello
I have managed to get the scraper.py to scrape, by running sudo python3.6 scraper.py and it is giving an output as below.
but how do I see this information in my local version of planningalerts, it appears to be saving it to a sqlite db ?
or is there something I'm missing ?

I have cloned https://github.com/43South/tasmaniada and am using this repo but I dont know how you get it to show in a local copy of planningalerts that I have running.

sudo python3.6 scraper.py
INFO:root:starting
INFO:root:breakoday: found 6 applications
INFO:root:brighton: found 3 applications
INFO:root:burnie: found 5 applications
ERROR:root:failed to run for centralcoast
ERROR:root:failed to run for centralhighlands
ERROR:root:failed to run for circularhead
ERROR:root:failed to run for dorset
ERROR:root:failed to run for flinders
INFO:root:georgetown: found 0 applications
INFO:root:glamorganspringbay: found 16 applications
INFO:root:glenorchy: found 2 applications
INFO:root:kentish: found 3 applications
ERROR:root:failed to run for kingborough
INFO:root:kingisland: found 0 applications
ERROR:root:failed to run for latrobe
ERROR:root:failed to run for meander
ERROR:root:failed to run for northernmidlands
INFO:root:tasman: found 11 applications
INFO:root:waratahwynyard: found 6 applications
ERROR:root:failed to run for westcoast
INFO:root:westtamar: found 9 applications
DEBUG:root:{'authority_label': 'breakoday', 'council_reference': 'DA 024-2024', 'address': '36 John Street, Cornwall, Tasmania, Australia', 'description': 'Residential - Retrospective Approval of Annex', 'info_url': 'https://www.bodc.tas.gov.au/wp-content/uploads/2018/06/Advert-Plans-DA-024-2024.pdf', 'date_scraped': '2024-04-03T15:02:50.284148', 'date_received': None, 'on_notice_from': None, 'on_notice_to': '2024-04-03'}
DEBUG:root:{'authority_label': 'breakoday', 'council_reference': 'DA 244-2023', 'address': '126 Scamander Avenue, Scamander, Tasmania, Australia', 'description': 'Residential - New Shed', 'info_url': 'https://www.bodc.tas.gov.au/wp-content/uploads/2018/06/Advert-Plans-DA-244-2023.pdf', 'date_scraped': '2024-04-03T15:02:50.284148', 'date_received': None, 'on_notice_from': None, 'on_notice_to': '2024-04-03'}
DEBUG:root:{'authority_label': 'breakoday', 'council_reference': 'DA 079-2023', 'address': 'King Street (Access over Crown Land, 2 Dunn Street) Mathinna, Tasmania, Australia', 'description': 'Residential - Retrospective Approval of Demolition of Dwelling, Dwelling Alterations & Additions', 'info_url': 'https://www.bodc.tas.gov.au/wp-content/uploads/2018/06/Advert-Plans-DA-079-2023.pdf', 'date_scraped': '2024-04-03T15:02:50.284148', 'date_received': None, 'on_notice_from': None, 'on_notice_to': '2024-04-16'}

Just wondering if anyone can give me a bit of a steer on this project. Keen to get all the Tasmanian Councils feeding into PlanningAlerts Website.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant