Can't Produce Segmentation Data

[skarakulak]

Hi,

I was trying to produce a segmentation data on an area in Philippines with the configuration below. label-maker download command creates the mbtiles and geojson files. However when I try to produce the labels and images, label-maker just creates empty folders. Where do I do wrong? I have installed mbviewer and made sure that the region that I downloaded contains labels.

{
  "country": "philippines",
  "bounding_box": [121.464843728242,18.312810843256784,121.50878904073498,18.354525525951416],
  "zoom": 12,
  "classes": [
    { "name": "Landuse", "filter": ["has", "landuse"] },
    { "name": "Roads", "filter": ["has", "highway"] },
    { "name": "Building", "filter": ["has", "building"] }
  ],
"imagery": "http://a.tiles.mapbox.com/v4/mapbox.satellite/{z}/{x}/{y}.jpg?access_token=pk.eyJ1Ijoic2thcmFrdWxhayIsImEiOiJjam54Z3YwaDYwMGhkM3FwY3BmeG1lZHd5In0.dAFYUvWpcw7katdB9xeBSg",
  "background_ratio": 1,
  "ml_type": "segmentation"
}

[drewbo]

@skarakulak just checked it out and I can confirm that I'm seeing the same issue. I'll mess around with this for a while and update soon. Thanks for the bug report!

[drewbo]

@skarakulak Looks like there is a TopologicalError thrown by one of the geometries and it silently causes the tile processing section to fail. I'll add some handling for that to fix this

[skarakulak]

Thanks a lot for the quick response, much appreciated

[skarakulak]

I have tried with the latest code but still getting no labels:

serkan@serkan-HP-ubuntu:~/temp_geo$ label-maker labels
Determining labels for each tile
TopologyException: Input geom 0 is invalid: Self-intersection at or near point 9 247 at 9 247
---
Landuse: 0 tiles
Roads: 0 tiles
Building: 0 tiles
Total tiles: 4
Writing out labels to data/labels.npz
serkan@serkan-HP-ubuntu:~/temp_geo$ label-maker images
Downloading 0 tiles to data/tiles

[drewbo]

@skarakulak it still isn't merged in yet. If you want to try it out now, you can install directly from the repo (pip install -e .) on the branch bug/118-skip-bad-geometries. Otherwise, we'll have a new release early next week

[NickElixir]

I got same problem.

TopologyException: Input geom 0 is invalid: Self-intersection at or near point 226 179 at 226 179

Analogical errors appear with other bounding boxes.
Is this bug or my mistake?

[NickElixir]

I tried the example from devseed.com and got same:
TopologyException: Input geom 0 is invalid: Self-intersection at or near point 216 84 at 216 84 The operation 'GEOSIntersection_r' could not be performed. Likely cause is invalidity of the geometry <shapely.geometry.polygon.Polygon object at 0x7fe07347d898> skipping
What's the problem?

[Geoyi]

@NickElixir, check your (multi)polygon/geometry, when it was created, some bow-tie/intersect was accidentally created. Check this - How to fix invalid polygon with self-intersection - Python .

[NickElixir]

@Geoyi, I don't understand what I should do. This is my bounding box: [1.09725, 6.05520, 1.34582, 6.30915] for togo

[skarakulak]

Hey Nick, It's been a while and I don't remember how I solved that particular issue. However I made some changes with the label-maker which worked for me in the end. Perhaps you might wanna go through the commits here https://github.com/skarakulak/label-maker Also you can clone the fork and try to get the data with your bounding box with this modified version. My solution was kind of a quick hack, and it changed the default behavior of label-maker which might result in undesired behavior in other tasks other than segmentation. So I didn't open a PR for it. Sorry for not being more helpful. Let me know if this doesn't solve the problem or if you have any questions regarding the changes that I have made in the fork. Best, Serkan

…

On Thu, Jun 13, 2019, 7:55 AM Niсk_Elixir ***@***.***> wrote: @Geoyi <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Geoyi&d=DwMCaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=9jC6quPZN0RnI866qaQhEg&m=3LqIGfEhS30CgGlFN9xm1Q0Ic7chmddXzKEkBcZsw8w&s=pJmOWHUvC4AxEQCdx3sHN_QxqZBXD1ngmuJ5RCOqmFo&e=>, I don't understand what I should do. This is my bounding box: [1.09725, 6.05520, 1.34582, 6.30915] — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_developmentseed_label-2Dmaker_issues_118-3Femail-5Fsource-3Dnotifications-26email-5Ftoken-3DAB6G5H3BTCBZHFKAFKLA3CDP2IYS5A5CNFSM4GCWR7J2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXTOJRY-23issuecomment-2D501671111&d=DwMCaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=9jC6quPZN0RnI866qaQhEg&m=3LqIGfEhS30CgGlFN9xm1Q0Ic7chmddXzKEkBcZsw8w&s=1C0-ljNxom4GpJJk1fjUSYf3YDwX4Z4pev3HQCW1QQA&e=>, or mute the thread <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AB6G5HZDRO5GD4VCGGSX3WDP2IYS5ANCNFSM4GCWR7JQ&d=DwMCaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=9jC6quPZN0RnI866qaQhEg&m=3LqIGfEhS30CgGlFN9xm1Q0Ic7chmddXzKEkBcZsw8w&s=tU8PcmWTf8kyRU1ASfjvfngOfrukBMDTGuQLGuT8eYQ&e=> .

[NickElixir]

@skarakulak I use Colab for downloading images for segmentation. How I can install your fork in Colab?

[drewbo]

@NickElixir @skarakulak @Geoyi the problematic geometries sometimes come directly from OSM or when they get chopped up by our tiling. It will show the warning but skips the geometry and should continue from there.

@NickElixir are you still getting results after seeing that warning?

[NickElixir]

@drewbo, I got some but less now. Does this error mean that I skip tags?
I see that dataset is not full and not balanced.
The problem is that probability of error per tag is not random.

[drewbo]

@NickElixir it's possible to fix the underlying geometries first then load as a GeoJSON but that's a bit of a pain. When you say that you're getting "less now" (I assume labels), less than when?