2478 add streets table from datagov

[ziv17]

No description provided.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 6.41026% with 73 lines in your changes missing coverage. Please review.

Project coverage is 53.20%. Comparing base (400af13) to head (24ea9d1).
Report is 52 commits behind head on dev.

Files	Patch %	Lines
anyway/parsers/streets.py	0.00%	62 Missing ⚠️
anyway/parsers/infographics_data_cache_updater.py	25.00%	6 Missing ⚠️
main.py	0.00%	5 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##              dev    #2668      +/-   ##
==========================================
- Coverage   53.22%   53.20%   -0.02%     
==========================================
  Files         119      120       +1     
  Lines        9924    10072     +148     
==========================================
+ Hits         5282     5359      +77     
- Misses       4642     4713      +71     

Flag	Coverage Δ
unittests	53.20% <6.41%> (-0.02%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[atalyaalon]

@ziv17 this limit is too small (and is it necessary to add a limit? perhaps we can just query all data?) since today there are 61455 records in the table, and I assume we can reach a time, at some point where 100000 is not enough.
If we must add a limit, I suggest adding one or two zeros to be on the safe side :)

[ziv17]

The default limit is 32k, so we must supply a limit. I did not find a way to work with pagination. I will set the limit to 1M.

[atalyaalon]

@ziv17 Great work! It's already merged to dev but not to master yet: I'm waiting for the CBS final answer that our method is correct for the merge.
In addition, before merging to production I suggest you also remove the code that updates the streets table from CBS data (this code is in cbs anyway/parsers/cbs/executor.py) - I opened an issue for this: #2674

However, let's wait for the final answer of the CBS and then implement this last part