Color problem when using different tiles providers?

[BastienFR]

I've noticed the the color used to with addGlifyPolygons doesn't work for all tile providers.

See RE:

# loading the libraries
library(sf)  
library(leaflet)
library(leaflet.glify)
library(colourvalues)

# preparing the shapefile
nc <- st_read(system.file("gpkg/nc.gpkg", package="sf"), quiet = TRUE) %>% 
  st_transform(st_crs(4326)) %>% 
  st_cast('POLYGON')

# setting the colors
cols = colour_values_rgb(nc$AREA, palette = "inferno", include_alpha = FALSE) / 255

# making the first map like in your example:
leaflet() %>%
  addProviderTiles(provider = providers$CartoDB.DarkMatter) %>%
  leaflet.glify:::addGlifyPolygons(nc, color = cols) %>% 
  fitBounds(lng1 = unname(st_bbox(nc)$xmin), lng2 = unname(st_bbox(nc)$xmax), lat1 = unname(st_bbox(nc)$ymin), lat2 = unname(st_bbox(nc)$ymax))

# making the second map with a different tile provider:
leaflet() %>%
  addProviderTiles(provider = providers$OpenStreetMap.Mapnik) %>%
  leaflet.glify:::addGlifyPolygons(nc, color = cols) %>% 
  fitBounds(lng1 = unname(st_bbox(nc)$xmin), lng2 = unname(st_bbox(nc)$xmax), lat1 = unname(st_bbox(nc)$ymin), lat2 = unname(st_bbox(nc)$ymax))

Do you know what the problem is?

[tim-salabim]

Can you provide your sessionInfo() please?
I see:

leaflet() %>%
  addProviderTiles(provider = providers$CartoDB.DarkMatter) %>%
  leaflet.glify:::addGlifyPolygons(nc, color = cols) %>% 
  fitBounds(lng1 = unname(st_bbox(nc)$xmin), 
            lng2 = unname(st_bbox(nc)$xmax),
            lat1 = unname(st_bbox(nc)$ymin),
            lat2 = unname(st_bbox(nc)$ymax))

# making the second map with a different tile provider:
leaflet() %>%
  addProviderTiles(provider = providers$OpenStreetMap.Mapnik) %>%
  leaflet.glify:::addGlifyPolygons(nc, color = cols) %>% 
  fitBounds(lng1 = unname(st_bbox(nc)$xmin), 
            lng2 = unname(st_bbox(nc)$xmax),
            lat1 = unname(st_bbox(nc)$ymin),
            lat2 = unname(st_bbox(nc)$ymax))

Session info (click to view)

R version 3.5.1 (2018-07-02)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

Matrix products: default

locale:
[1] LC_COLLATE=German_Germany.1252  LC_CTYPE=German_Germany.1252    LC_MONETARY=German_Germany.1252
[4] LC_NUMERIC=C                    LC_TIME=German_Germany.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] colourvalues_0.1.1  leaflet.glify_0.1.0 leaflet_2.0.2       sf_0.7-0           

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.18    magrittr_1.5    units_0.6-1     xtable_1.8-3    R6_2.2.2        tools_3.5.1     grid_3.5.1     
 [8] e1071_1.7-0     DBI_1.0.0.9000  rapidjsonr_1.1  htmltools_0.3.6 crosstalk_1.0.0 class_7.3-14    yaml_2.2.0     
[15] digest_0.6.17   shiny_1.1.0     spData_0.2.9.3  later_0.7.3     htmlwidgets_1.2 promises_1.0.1  mime_0.5       
[22] geojsonsf_1.2   compiler_3.5.1  classInt_0.2-3  jsonlite_1.5    httpuv_1.4.5

[BastienFR]

R version 3.4.4 (2018-03-15)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.4 LTS

Matrix products: default
BLAS: /usr/lib/atlas-base/atlas/libblas.so.3.0
LAPACK: /usr/lib/atlas-base/atlas/liblapack.so.3.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8   
 [6] LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] colourvalues_0.1.1  leaflet.glify_0.1.0 leaflet_2.0.2       sf_0.7-2           

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.19    magrittr_1.5    units_0.6-1     xtable_1.8-2    R6_2.3.0        tools_3.4.4     grid_3.4.4      e1071_1.6-8     DBI_1.0.0      
[10] rapidjsonr_1.1  htmltools_0.3.6 crosstalk_1.0.0 class_7.3-14    yaml_2.2.0      digest_0.6.18   shiny_1.1.0     spData_0.2.8.3  later_0.7.2    
[19] htmlwidgets_1.3 promises_1.0.1  mime_0.5        geojsonsf_1.2   compiler_3.4.4  classInt_0.2-3  jsonlite_1.5    httpuv_1.4.3   

Working on Rstudio server

[BastienFR]

Considering you didn't have the problem, I did more test on my side. I've tried to use this code on my windows machine. Actually, in that case, the result is even worst, I get nothing display for the first map and only the tiles with no polygons for the second call. here is my sessionInfo for me Windows machine. All my packages are up to date but R is 3.5.0.

R version 3.5.0 (2018-04-23)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows Server >= 2012 x64 (build 9200)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                           LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] colourvalues_0.1.1  leaflet.glify_0.1.0 leaflet_2.0.2       sf_0.7-1           

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.19    magrittr_1.5    units_0.6-1     xtable_1.8-3    R6_2.3.0        tools_3.5.0     grid_3.5.0     
 [8] e1071_1.7-0     DBI_1.0.0       rapidjsonr_1.1  htmltools_0.3.6 crosstalk_1.0.0 class_7.3-14    yaml_2.2.0     
[15] digest_0.6.18   shiny_1.1.0     spData_0.2.9.4  later_0.7.5     htmlwidgets_1.3 promises_1.0.1  mime_0.6       
[22] geojsonsf_1.2   compiler_3.5.0  classInt_0.2-3  jsonlite_1.5    httpuv_1.4.5  

[tim-salabim]

Are your problems in RStudio viewer, the browser or both? And which browser are you using? Chrome does normally not allow local files to be served, so nothing may be displayed. Firefox usually works fine.
Regarding your second problem, can you please open the map in a browser (Firefox) and post a screenshot of the console (press ctrl + shift + i)?

[BastienFR]

I'm continuing my tests both on windows and linux. On windows:

• Updated everything (all packages, R and Rstudio): Same problem

• Close Rstudio an ran the script in Base R gui and open the html with both Firefox and Chrome (both Firefox and Chrome gave the same results):

  • Openstreet map:
    

  • CartoDB:
    

• In this case, both tiles show up (which is not the case in Rstudio where only the OpenStreet tiles show up). However, I still don't get my actual shape (nc). This may be unrelated to the original problem that I posted as it do not seems link with the colors. If I remove the colors=cols argument, I still don't see my shape.

For the problem on my linux machine, I'm currently updating my machine and it's taking a long time. I'll post my results as soon as I have them.

[tim-salabim]

Thanks! Can you please also provide screeshots with those that show the console, not the inspector.

[BastienFR]

oups, sorry...

hope it's ok this time­...

[BastienFR]

For Linux: I'm on rstudio Server, running on a azure virtual machine with ubuntu 16.04 and called by Chrome on my local windows machine. I can't use firefox on this machine, corporate policy... I've updated all R and the packages but I didn't update my Rstudio server. If I run the script, I get the same thing as before the update. see below:

• Openstreet map:

  • before the console:
    
  • After the console (when I open the console, my map disappear...):
    

• for cartoDB:

  • before the console:
    

  • after the console:
    

[tim-salabim]

Perfect thanks!
Now can you post a screenshot of the (beginning of) "glpolygons_data.json" mentioned in the first XML-parsing error? You can just navigate to that folder and open it in the browser or press ctrl+u in the tab with the map. A new tab should open where you should see a link to that file - just click on it

[BastienFR]

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8"/>
<script src="lib/htmlwidgets-1.3/htmlwidgets.js"></script>
<script src="lib/jquery-1.12.4/jquery.min.js"></script>
<link href="lib/leaflet-1.3.1/leaflet.css" rel="stylesheet" />
<script src="lib/leaflet-1.3.1/leaflet.js"></script>
<link href="lib/leafletfix-1.0.0/leafletfix.css" rel="stylesheet" />
<script src="lib/Proj4Leaflet-1.0.1/proj4-compressed.js"></script>
<script src="lib/Proj4Leaflet-1.0.1/proj4leaflet.js"></script>
<link href="lib/rstudio_leaflet-1.3.1/rstudio_leaflet.css" rel="stylesheet" />
<script src="lib/leaflet-binding-2.0.2/leaflet.js"></script>
<script src="lib/leaflet-providers-1.1.17/leaflet-providers.js"></script>
<script src="lib/leaflet-providers-plugin-2.0.2/leaflet-providers-plugin.js"></script>
<script src="lib/addGlifyPoints-0.0.1/addGlifyPoints.js"></script>
<script src="lib/addGlifyPoints-0.0.1/addGlifyPolygons.js"></script>
<script src="lib/addGlifyPoints-0.0.1/glify.js"></script>
<script src="lib/addGlifyPoints-0.0.1/src/js/canvasoverlay.js"></script>
<script src="lib/addGlifyPoints-0.0.1/src/js/gl.js"></script>
<script src="lib/addGlifyPoints-0.0.1/src/js/index.js"></script>
<script src="lib/addGlifyPoints-0.0.1/src/js/map-matrix.js"></script>
<script src="lib/addGlifyPoints-0.0.1/src/js/points.js"></script>
<script src="lib/addGlifyPoints-0.0.1/src/js/shapes.js"></script>
<script src="lib/addGlifyPoints-0.0.1/src/js/utils.js"></script>
<script src="lib/addGlifyPoints-0.0.1/src/shader/fragment/dot.glsl"></script>
<script src="lib/addGlifyPoints-0.0.1/src/shader/fragment/point.glsl"></script>
<script src="lib/addGlifyPoints-0.0.1/src/shader/fragment/polygon.glsl"></script>
<script src="lib/addGlifyPoints-0.0.1/src/shader/fragment/puck.glsl"></script>
<script src="lib/addGlifyPoints-0.0.1/src/shader/fragment/simple-circle.glsl"></script>
<script src="lib/addGlifyPoints-0.0.1/src/shader/fragment/aquare.glsl"></script>
<script src="lib/addGlifyPoints-0.0.1/src/shader/vertex/default.glsl"></script>
<link id="glpolygonsdt-1-attachment" rel="attachment" href="lib/glpolygonsdt-1/glpolygons_data.json"/>
<link id="glpolygonscl-1-attachment" rel="attachment" href="lib/glpolygonscl-1/glpolygons_color.json"/>

</head>
<body style="background-color:white;">
<div id="htmlwidget_container">
  <div id="htmlwidget-6895209d5e1286c1bf2f" style="width:100%;height:400px;" class="leaflet html-widget"></div>
</div>
<script type="application/json" data-for="htmlwidget-6895209d5e1286c1bf2f">{"x":{"options":{"crs":{"crsClass":"L.CRS.EPSG3857","code":null,"proj4def":null,"projectedBounds":null,"options":{}}},"calls":[{"method":"addProviderTiles","args":["CartoDB.DarkMatter",null,null,{"errorTileUrl":"","noWrap":false,"detectRetina":false}]},{"method":"addGlifyPolygons","args":["glpolygonsdt","glpolygonscl",null,0.6,10]}],"fitBounds":[33.8821646134549,-84.3237734916375,36.5897678106392,-75.4565804266542,[]]},"evals":[],"jsHooks":[]}</script>
<script type="application/htmlwidget-sizing" data-for="htmlwidget-6895209d5e1286c1bf2f">{"viewer":{"width":"100%","height":400,"padding":0,"fill":true},"browser":{"width":"100%","height":400,"padding":0,"fill":true}}</script>
</body>
</html>

Is that it?

[BastienFR]

The other option is:

[tim-salabim]

can you just click on link id="glpolygonsdt-1-attachment" rel="attachment" href="lib/glpolygonsdt-1/glpolygons_data.json" in this tab please. The href part should be clickable

[BastienFR]

`{"type":"FeatureCollection","features":[{"type":"Feature","properties":{"id":1},"geometry":{"type":"Polygon","coordinates":[[[-81.47257911591413,36.23448491106877],...

[tim-salabim]

Did you put that first ` there? Or is this part of what you see in the browser? A screenshot would be better

[BastienFR]

I don't see the ` you are talking about.

[tim-salabim]

Ok, I can reproduce the error. It happens when I open the index.html directly from the RTemp... folder. If I simply set options(viewer = NULL) or click Open in browser in RStudio then I see the data. Have you tried these options?

[BastienFR]

When adding options(viewer = NULL), the behaviour varies on both my machine and depending the tool use:

• windows:

  • rstudio: doesn't work, I get an IT McAfee bug which my TI guy just told me is link with a proxy issue. This problem is on my side so I'll try to fix it myself
  • Base R Gui: Directly open in the browser, I see the base map, and not my shape, so the problem stated above.
  • (Note if you are interested: the difference between Rstudio and Rgui in that case is that rstudio create a webpage localhost while R GUI create a file on disc)

• linux:

  • studio server: I see the tiles and the shape, but the colors are wrong like in the first post of this thread.

[tim-salabim]

Ok, I see the same behaviour when using R GUI ("file:///") which seems to have problems with relative attachments.
Can you try

m = leaflet() %>%
  addProviderTiles(provider = providers$CartoDB.DarkMatter) %>%
  leaflet.glify:::addGlifyPolygons(nc, color = cols) %>% 
  fitBounds(lng1 = unname(st_bbox(nc)$xmin), 
            lng2 = unname(st_bbox(nc)$xmax),
            lat1 = unname(st_bbox(nc)$ymin),
            lat2 = unname(st_bbox(nc)$ymax))

mapshot(m, url = "some/file/on/your/disk.html", selfcontained = FALSE)

and try to open that in firefox. This should diplay the shapes.

As for your RStudio server problem, would you mind trying to set opacity = 1, I think what we have here is a problem with the shader...

[BastienFR]

For the second part (rstudio server), you got it! It's a problem with opacity: See my example below:

with no opacity setting:

with opacity=1:

with opacity = 0.999

with opacity = 0.9

Weird behaviour!

[BastienFR]

For the mapshot option on windows, I still don't see anything after running:

mapview::mapshot(m, url = "D:/temp/test.html", selfcontained = FALSE)

The file produced do not have polygons. Seems to me like were are troubleshooting to distinct problems...

[tim-salabim]

Thanks for testing, I have seen the weird opacity behaviour before, but didn't encounter it recently. No idea what changed though... strange

For the url part in mapshot just provide a path where you want to save the map, like C:/testing/mymap.html and then open mymap.html

[BastienFR]

Yeah, just figure after doing ?mapshot... see my edited comment...

[BastienFR]

New update: Good news, I fixed my proxy problem and I can reproduce the color/opacity bug on windows/Rstudio. So in that case, if I call in the viewer pane, only CartoBD works and OpenStreet map doesn't (kind of weird behaviour...). If I use options(viewer = NULL), than both open in the browser and the shape is visible, but the colors are wrong if I don't manage the opacity level.

However, I still do not see my shape if I call the script in the R GUI.

[tim-salabim]

I think I found the culprit regarding the missing shapes

https://developer.mozilla.org/en-US/docs/Archive/Misc_top_level/Same-origin_policy_for_file:_URIs

I will investigate

[tim-salabim]

@BastienFR the above commit should fix the issue with no shapes being shown. The "funny colour" issue is likely beyond this repo, but may need to be addressed in the downstream JS dependency.

[BastienFR]

Good, the shape is being shown now no problem! Thanks. For the colors, I'll try to manage around the tiles selection and opacity to find something that works for me. Hopefully the problem will get fix some time. Thanks for your time and your great package.

[tim-salabim]

I just found that in Opera the colors seem to be fine, no matter the background. Do you have any chance to test and confirm this?

[BastienFR]

sadly I get the same behaviour:

And while doing this test, I've notice I lose the shape when I resize my window and move it around

P.S. That test was done on windows.

[tim-salabim]

bummer! I will keep investigating... The shape should reappear once you interact with the map

[BastienFR]

To add to the whole confusion... I have access today to another computer with windows 10 this time. The main difference with this computer vs the other ones used above is that it's my own computer not on a corporate network (actually Azure Machines within my company firewall). Other things could maybe have changes (mostly version of the different software), but that time I get it right on both Chrome and Firefox:

So, could it be some king of security issue? It would be weird considering that it's linked with the opacity level. Also, we never had any problem with the base leaflet with that setup, why would leaflet.glify be different?

But now that I'm thinking about it, I think our Azure machine are very limited graphic wise while my personal machine has a Nvidia 1060. Could the GL part of the code of your software requires more graphical computing power? Adding the opacity would complicate the rendering? However, why would it work with CartoDB and the dark background... Could the fact that my machine are Virtual caused the problem?

I'm sending that out there, but this is way over my level.

[tim-salabim]

Over my skill level too unfortunately. BTW, I cannot access the html which is a localhost...

[BastienFR]

For the local host, it was a copy-paste mistake, I've fixed it

[BastienFR]

So after talking to some people more aware of our server, it doesn't seem the graphical capacity of our system is the problem. Actually, to be sure I've build the map on rstudio linux server, saved it with mapshot, send it by email to my computer where it works and opened it there and no luck, I had no shape displayed. very weird.

[tim-salabim]

Can you email me the map?

[BastienFR]

https://www.dropbox.com/sh/icbll2annfo32ar/AADDhf3NfCyRIngTieijKkiLa?dl=0

In the dropbox link is the problematic map. I actually joined 2 maps:

The first: test_glify, I’ve created it on Rstudio Server on a Linux Azure VM and save it using mapview::mapshot. I couldn't view it on either of my computer

The second: test_glify_works , I’ve created it on my personal windows machine, it worked there.

Try them both, you’ll notice that there is a huge file difference. Like the polygons weren’t saved with the problematic one. The same code was used to save both.

[BastienFR]

Ok, Time out. I think the problem of shape not displayed is because I've updated the package only on my windows machine and not my linux one (I'm getting confuse with all the tests I'm doing, that's a bad sign...) So in the dropbox link, it's probably the same bug you already fixed. However I can't confirm it because I exceeded (???) my github api limit... I'll could retest it tomorrow.

I redid the same test but on my Azure Windows machine which is updated for your package. I've added a folder in the dropbox link (test2) with the result. Interestingly, these 3 maps, which are not working on 2 of my machines are all working find on my personal machine... Therefore, it seems like it's pointing to again some kind of remote rendering issue... Tomorrow, I'll get a couple of colleagues to test the visualisation on there side as well.

sorry about that

[BastienFR]

I did more and more tests and I think I found the root of it. It's quite an obscure problem.

The problem of bad colors when opacity<1 is caused by the interaction of Chrome and using a virtual machine. When I open the problematic map in Chrome in remote session (tested on Citrix and Azure) using Chrome, it fails. If I use Firefox it works even when on a remote machine. Using chrome on a local machine works all the time.

Now, for an obscure reason, this doesn't apply if we use providers$CartoDB.DarkMatter which work on Chrome on a remote desktop. However, I've tested almost all providers and that was the only one working. Another seem to work but as soon as I zoomed the colors broke,

Now, clearly this is out of the repo, however I still think is worth documenting it here.

The only possible workaround I've found is to use opacity=1 but adding the baselayer label over it. This adds the cities, bot not the streets and territory borders so it's less than ideal. https://gis.stackexchange.com/questions/285687/r-leaflet-bring-base-layer-labels-to-front

Hopefully a chrome or webgl update will fix that soon.

If anybody sees that and have access to other kind of remote desktop (other than Azure or Citrix), it would be nice to know if they are experiencing the same problem.

[tim-salabim]

Reopening, as I still see this on my laptop w ubuntu. Chrome, Firefox show wrong colors, Opera and RStudio viewer are fine...