Add function for #154 #155

Robinlovelace · 2020-12-16T18:52:25Z

No description provided.

agila5 · 2020-12-16T20:28:50Z

Hi Robin! Thanks for the PR. I checked the review and I think it's quite positive 🎉. Anyway, I'm a little bit busy in the next days but I will work on osmextract during the weekend (or, anyway, as soon as possible).

Robinlovelace · 2020-12-16T21:47:17Z

This is ready for review @agila5, no rush but it seems to work. Output is very chatty atm which is no bad thing I think, feedback welcome on idea and implementation:

ghent = oe_get("ghent")
Warning: The input place was matched with multiple geographical zones: Ghana - Kent. Selecting the first match.
No exact matching found for place = ghent. Best match is Ghana.
String distance between best match and the input place is 2, while the maximum threshold distance is equal to 1. You can try increasing the max_string_dist parameter, look for a closer match in the chosen provider database or consider using a different match_by variable.
No match in the OSM provider data. Searching for the location online
trying URL 'https://nominatim.openstreetmap.org/search?q=ghent&limit=1&format=geojson'
downloaded 591 bytes

Got data for: Gent, Oost-Vlaanderen, België / Belgique / Belgien, c(3.7250121, 51.0538286)
although coordinates are longitude/latitude, st_intersects assumes that they are planar
The input place was matched with multiple geographical areas. Selecting the areas with the highest "level". See the help page associated to the chosen provider for an explanation of the meaning of the "level" field
trying URL 'https://download.geofabrik.de/europe/belgium-latest.osm.pbf'
Content type 'application/octet-stream' length 418022088 bytes (398.7 MB)
==================================================
downloaded 398.7 MB

File downloaded!
Start with the vectortranslate operations on the input file!
0...10...20...30...40...50...60...70...80...90...100 - done.
Finished the vectortranslate operations on the input file!
Reading layer `lines' from data source `/mnt/57982e2a-2874-4246-a6fe-115c199bc6bd/data/osm/geofabrik_belgium-latest.gpkg' using driver `GPKG'
Simple feature collection with 1486676 features and 9 fields
geometry type:  LINESTRING
dimension:      XY
bbox:           xmin: -5.652017 ymin: 49.43529 xmax: 8.589302 ymax: 62.51466
geographic CRS: WGS 84

agila5 · 2020-12-24T15:54:48Z

Hi @Robinlovelace! Checking it right now. I was thinking of a slightly different workflow, but I will add some code for testing/discussing in a few hours.

agila5

Hi @Robinlovelace! I just pushed a commit for this PR. I slightly changed your approach in two ways: 1) the oe_search() function now is in oe_match() since it could be useful outside of oe_get() and 2) oe_match() now checks the other providers before using oe_serch(). So, for example,

library(osmextract)
oe_match("Leeds")
#> No exact match found for place = Leeds and provider = geofabrik. Best match is Laos. 
#> Checking the other providers.
#> I found an exact string match using provider = bbbike so I'm going to return that.
#> $url
#> [1] "https://download.bbbike.org/osm/bbbike/Leeds/Leeds.osm.pbf"
#> 
#> $file_size
#> [1] 19376705
oe_match("Olginate")
#> No exact match found for place = Olginate and provider = geofabrik. Best match is Suriname. 
#> Checking the other providers.
#> No exact match found in any OSM provider data. Searching for the location online.
#> although coordinates are longitude/latitude, st_intersects assumes that they are planar
#> The input place was matched with multiple geographical areas. Selecting the areas with the highest "level". See the help page associated to the chosen provider for an explanation of the meaning of the "level" field
#> $url
#> [1] "https://download.geofabrik.de/europe/italy/nord-ovest-latest.osm.pbf"
#> 
#> $file_size
#> [1] 416306623

^{Created on 2020-12-28 by the reprex package (v0.3.0)}

Now I need to review examples and vignettes and add some tests

Robinlovelace · 2020-12-28T19:51:11Z

Hi @Robinlovelace! I just pushed a commit for this PR. I slightly changed your approach in two ways: 1) the oe_search() function now is in oe_match() since it could be useful outside of oe_get() and 2) oe_match() now checks the other providers before using oe_serch().

Great thinking on including it in oe_match() 👍

Robinlovelace · 2020-12-28T19:54:59Z

Another thought: worth using oe_match() in oe_download() so that if you give it something other than a URL it still tries to find something useful?

Fix #156

agila5 · 2020-12-28T22:13:27Z

Another thought: worth using oe_match() in oe_download() so that if you give it something other than a URL, it still tries to find something useful?

I would like to keep the two operations in two distinct functions since if you want just to download PBF/GPKG data for a given provider, then you can run oe_get(input, download_only = TRUE) (adding skip_vectortranslate if you are not interested in the GPKG conversion). If you want, we can create a wrapper around oe_get with those parameters as default, not sure about the name.

agila5 · 2021-01-01T17:45:13Z

Sometimes next week I will finish re-reading the vignette and then I think it will be ready for review/merge!

Robinlovelace · 2021-01-08T08:34:59Z

I think this is almost ready to go. Let me know if you want me to add something to the vignette. I've already used the new functionality in my work, useful for sure!

agila5 · 2021-01-08T10:36:32Z

Hi Robin! Thanks for the comment. If you can test this PR and find any problem, please report it. I plan to finish the PR on Monday 11/01 and then start with #160

Robinlovelace · 2021-01-08T10:45:18Z

Thanks @agila5, sounds good to me!

agila5 · 2021-01-11T22:18:09Z

Hi @Robinlovelace, I think it's ready for review.

agila5 · 2021-01-13T12:43:38Z

Hi @Robinlovelace, sorry to bother you. Can you check the PR today or tomorrow morning? Tomorrow (after lunch) I would like to complete the other PRs and write the answer for rOpenSci review.

Robinlovelace

Looks good!

Robinlovelace · 2021-01-13T12:54:49Z

R/download.R

@@ -36,10 +36,10 @@
 #' @export
 #'
 #' @examples
-#' its_match = oe_match("ITS Leeds", provider = "test")
+#' its_match = oe_match("ITS Leeds", provider = "test", quiet = TRUE)


Robinlovelace · 2021-01-13T12:55:21Z

R/get.R

 #' west_yorkshire = oe_get("West Yorkshire", quiet = FALSE)
+#' # Match with place name
+#' oe_get("Milan") # Warning: the .pbf file is 400MB


Great example!

Robinlovelace · 2021-01-13T12:55:39Z

R/match.R

+
+    # 2. Check the other providers and, if there is an exact match, just return
+    # the matched value from the other provider:
+    other_providers = setdiff(oe_available_providers(), provider)


Robinlovelace · 2021-01-13T12:55:57Z

R/match.R

@@ -367,7 +427,7 @@ oe_match_pattern = function(

  # Then we extract only the elements of the match_by_column that match the
  # input pattern.
-  match_ID = grep(pattern, match_by_column)
+  match_ID = grep(pattern, match_by_column, ignore.case = TRUE)


Robinlovelace · 2021-01-13T12:56:31Z

R/zzz.R

-    "Geofabrik data are taken from https://download.geofabrik.de/",
-    "For usage details of bbbike data see https://download.bbbike.org/osm/",
-    "OpenStreetMap_fr data are taken from http://download.openstreetmap.fr/",
+    "Check the package website for more details.",


One suggestion: Add the link to the website.

Add function for #154

7a8da45

Robinlovelace added 2 commits December 16, 2020 21:16

Document oe_search

cbdb80e

Close #154

673fb58

Robinlovelace marked this pull request as ready for review December 16, 2020 21:42

Robinlovelace requested a review from agila5 December 16, 2020 21:42

A slightly different approach for #155 + fix tests

c5cebed

agila5 approved these changes Dec 28, 2020

View reviewed changes

agila5 marked this pull request as draft December 28, 2020 17:26

Andrea Gilardi added 2 commits December 28, 2020 19:27

update existing tests

09e12a9

new tests for the interaction between oe_match and oe_search

72bddbc

Andrea Gilardi added 2 commits December 28, 2020 22:33

Change startup message

07d6544

Fix #156

update docs

246e06b

Andrea Gilardi added 7 commits December 28, 2020 23:19

oops

20cbe3a

update docs

f53f965

update README

3c33004

add note

2222280

oe_search should be used only when match_by == name

dc64469

add tests

ceb1147

review vignette pt1

e65008f

agila5 mentioned this pull request Jan 5, 2021

Submission: osmextract ropensci/software-review#395

Closed

31 tasks

update vignette

3003a37

agila5 marked this pull request as ready for review January 11, 2021 22:17

Update zzz.R

bd3e8b2

Robinlovelace commented Jan 13, 2021

View reviewed changes

Robinlovelace merged commit 7aaaa3c into master Jan 13, 2021

agila5 deleted the oe_search branch January 14, 2021 14:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add function for #154 #155

Add function for #154 #155

Robinlovelace commented Dec 16, 2020

agila5 commented Dec 16, 2020

Robinlovelace commented Dec 16, 2020

agila5 commented Dec 24, 2020

agila5 left a comment

Robinlovelace commented Dec 28, 2020

Robinlovelace commented Dec 28, 2020

agila5 commented Dec 28, 2020

agila5 commented Jan 1, 2021

Robinlovelace commented Jan 8, 2021

agila5 commented Jan 8, 2021

Robinlovelace commented Jan 8, 2021

agila5 commented Jan 11, 2021

agila5 commented Jan 13, 2021

Robinlovelace left a comment

Robinlovelace Jan 13, 2021

Robinlovelace Jan 13, 2021

Robinlovelace Jan 13, 2021

Robinlovelace Jan 13, 2021

Robinlovelace Jan 13, 2021

Add function for #154 #155

Add function for #154 #155

Conversation

Robinlovelace commented Dec 16, 2020

agila5 commented Dec 16, 2020

Robinlovelace commented Dec 16, 2020

agila5 commented Dec 24, 2020

agila5 left a comment

Choose a reason for hiding this comment

Robinlovelace commented Dec 28, 2020

Robinlovelace commented Dec 28, 2020

agila5 commented Dec 28, 2020

agila5 commented Jan 1, 2021

Robinlovelace commented Jan 8, 2021

agila5 commented Jan 8, 2021

Robinlovelace commented Jan 8, 2021

agila5 commented Jan 11, 2021

agila5 commented Jan 13, 2021

Robinlovelace left a comment

Choose a reason for hiding this comment

Robinlovelace Jan 13, 2021

Choose a reason for hiding this comment

Robinlovelace Jan 13, 2021

Choose a reason for hiding this comment

Robinlovelace Jan 13, 2021

Choose a reason for hiding this comment

Robinlovelace Jan 13, 2021

Choose a reason for hiding this comment

Robinlovelace Jan 13, 2021

Choose a reason for hiding this comment