Bing loads old and low-res imagery < z20

[boothym]

Today when editing I noticed that the usual Bing imagery was not showing unless I zoomed in to z20.

If you look at an area of London with very clear and recent (2020) imagery - https://www.openstreetmap.org/edit#map=20/51.47330/-0.11435 - it appears at zoom 20. However before today it would also display on other zoom levels, instead it now shows low-res and much older (2011) ESRI world (clarity) when zooming out.

I see that iD has been updated to 2.1.0 on osm.org, and in the changelog that it mentions some updates to Bing imagery from #9133. Does this have something to do with the issue?

[mbrzakovic]

Thanks for bringing this to our attention. Working on providing the answer.

[SK53]

Just to confirm this is widespread. Latest Bing imagery only effectively available at zoom 20 (various places in British Isles). The older imagery which is displayed at lower zoom appears to be ESRI Clarity.

[kesterlester]

I would just like to leave a "me too" relating to this. My mapping is mostly of underground structures (gas pipelines, typically) in the UK, and this requires a lot of zooming in and out to detect crop markings, much of it wide angle as pipes are LARGE structures. Finding them has become > 10 times harder for me (and in some cases impossible) following this week's change as I can no longer compare Bing imagery at a wide variety of zoom levels.

[boothym]

@tyrasd you've added the waitfor and question labels - what further info do you need or are we waiting to figure out whether the PR linked in the OP is the cause of this issue?

The information about Bing vintage that used to show up in the background panel is also missing.

[simonpoole]

Just as a data point: no issues with Vespucci that I could see, current 2022 Maxar imagery here, just as on bing.com/maps.

This commit 66c5007 changes the meta data API url for iD.

Testing with the Vespucci Bing key shows that this doesn't return the detailed provider information that the previous url did and I get old imagery just as iD users do using the provided image url. However the weird thing is that the only difference in the ImageUrl returned is a pr=odbl tacked on the end, this is what is obviously causing different imagery to be returned.

New meta data:

Old meta data:

PS: @mbrzakovic if Bing/MS can't provide the current imagery on the previous ODbL compatible terms, it would be really important that this is communicated to all editor maintainers, so at least @brycenesbitt , @systemed and the JOSM developers.

[boothym]

Ok so now the high res Bing imagery at z20 in the location linked to in the OP has also disappeared from iD. This is pretty annoying as two weeks ago I had really good and fairly recent Bing imagery to use when editing in the UK 😕

If this is any help, I see in the dev tools console there is a 403 error:

https://dev.virtualearth.net/REST/v1/Imagery/Metadata/AerialOSM?include=ImageryProviders&uriScheme=https&key=Auk3J0jR9g1_PVQgdmL95zCOKVOc8g-FGq5Zgb5ik7w1Ri5SRyWILV-kksgbw-Gh

{"authenticationResultCode":"DeniedCredentials","brandLogoUri":"http://dev.virtualearth.net/Branding/logo_powered_by.png","copyright":"Copyright © 2022 Microsoft and its suppliers. All rights reserved. This API cannot be accessed and the content and any results may not be used, reproduced or transmitted in any manner without express written permission from Microsoft Corporation.","errorDetails":["The request was forbidden. Your credentials may be denied or suspended."],"resourceSets":[],"statusCode":403,"statusDescription":"Forbidden","traceId":"17006db14d144394be526ffcfabbf209|DU0000276A|0.0.0.1"}

[simonpoole]

@boothym likely the key is invalid and iD using the failover url that it has defined.

The main thing is it would be nice to know from MS/Bing what is actually going on so that we can address the valid concerns @SK53 has, outside of that Bing is under no obligation to provide any specific quality of imagery or any at all for OSM use, so you may simply have to use other sources.

[boothym]

so you may simply have to use other sources.

Of course, but it would be nice to know from anyone what's going on, just a bit strange as two weeks ago things were fine before an iD update, and now even z20 Bing has changed to using ESRI clarity.

Edit: just opened up JOSM and it seems like Bing loads up the recent high res imagery no problem, so this would seem to be an iD issue?

[SK53]

I think a minimum action would be to make ESRI the default imagery for the UK and push Bing down the stack.

Currently, the overall lack of complaints suggests to me that we have relatively few really active mappers using iD in the UK. However, it will significantly impact newer mappers, who will either abandon OSM because of 'useless' imagery or make inadvertently erroneous edits.

@boothym Yes imagery is fine in other editing tools where I've checked. Also agree z20 has gone (it disappeared as I pulled images for my diary post).

@simonpoole ah! a failover makes a lot of sense

[nc011]

and now even z20 Bing has changed to using ESRI clarity.

Ah this explains it. I've been getting weird artefacts with editing at z20 where some blocks are the high resolution Bing imagery and others aren't. I guess my browser has cached the high res tiles (when opening an "in private" session the high res has completely gone).

[boothym]

Currently, the overall lack of complaints suggests to me that we have relatively few really active mappers using iD in the UK. However, it will significantly impact newer mappers, who will either abandon OSM because of 'useless' imagery or make inadvertently erroneous edits.

Not sure that's the case - the lack of complaints probably says more about people's lack of knowledge of where to complain (i.e they've got to find this github or mailing lists etc)

Personally I've found ways around it recently by zooming in to z20 when I needed high res Bing, also depends on what you are editing as unless you're tracing buildings for example you may well manage without it. Though I've also done less editing, so indeed new mappers or those who don't know workarounds will be the most affected.

And people who are more serious and do know where to complain, might not be affected by the issue at all if they are using JOSM.

[tyrasd]

@tyrasd you've added the waitfor and question labels - what further info do you need or are we waiting to figure out whether the PR linked in the OP is the cause of this issue?

the question label indicates that this is not something directly actionable by changing the iD code itself, and the waitfor label is to indicate that we're waiting to see if @mbrzakovic can find out what is going on behind the scenes.

[boothym]

Don't know if this will help troubleshooting (fallback?), but I notice that the north of Scotland - which used to have unusable Bing imagery before 2020 - still has the high res imagery at z18, see: https://www.openstreetmap.org/edit?way=857334989#map=18/57.51857/-4.21413 (go south west and you will move from high res to poorer quality Bing)

[boothym]

As a workaround, if you still want to use iD with current Bing imagery, the RapiD editor seems to work fine with it: https://mapwith.ai/rapid

[reubn]

OSM has a licence for Bing imagery.

This a case of fixing the incorrect credentials in the metadata request, thus avoiding the fallback URL and ODbL fallback imagery.

The licence for Bing imagery makes no mention of ODbL.

[reubn]

The quickest workaround is to just revert to the old Bing URL. This can be done by using the 'custom', option in the iD backgrounds menu and using https://ecn.t{switch:0,1,2,3}.tiles.virtualearth.net/tiles/a{u}.jpeg?g=587&n=z

There are no issues regarding copyright and licencing as OSM have a license, as mentioned above!

[simonpoole]

OSM has a licence for Bing imagery.

OSM has special permission to use Bing imagery in ways not allowed by others. It does not make any representations as to which imagery will be provided, but it is contingent on us using the provided (by Bing) credentials.

This a case of fixing the incorrect credentials in the metadata request, thus avoiding the fallback URL and ODbL fallback imagery.

The licence for Bing imagery makes no mention of ODbL.

The previous permission did, and, as that fact implies, the situation can change (again), which is why I was asking for clarification.

[reubn]

Who manages OSM's agreement with Microsoft? @simonpoole

[simonpoole]

Who manages OSM's agreement with Microsoft? @simonpoole

Nobody, that's part of the problem. Conceivably the EWG could do it with input from the LWG (that would avoid similar issues we are currently having with the Maxar api keys, were the contacts has gone AWOL).

[tyrasd]

#9197 documented another side effect likely connected to the same root cause as this issue: vintage is not available at the moment for the bing maps key we have to use in iD. 😒

[arch0345]

vintage is not available at the moment for the bing maps key we have to use in iD. 😒

Vintage seems to work now:

[tyrasd]

OK. Still, iD only loads the lower resolution tiles. Now, the only difference is the additional pr=odbl parameter which is supplied by the endpoint used in iD. Here's a comparison of the same tile requested witht the pr=odbl parameter (left) and without it:

iD is getting this pr=odbl parameter from the metadata endpoint https://dev.virtualearth.net/REST/v1/Imagery/Metadata/AerialOSM?include=ImageryProviders&key=… (note the OSM in AerialOSM), while other editors such as JSOM or rapiD continue to use the metadata endpoint https://dev.virtualearth.net/REST/v1/Imagery/Metadata/Aerial?include=ImageryProviders&key=… (without OSM in the URL) which results in tile URLs without the pr=odbl parameter.

Now, my question is: Is iD using the "wrong" endpoint, and could switch to the …/Metadata/Aerial/… (like is used by JOSM or rapiD), or are JOSM & co using the wrong endpoint and should be updated? @mbrzakovic can you help us getting this question resolved?

[ninoslavp]

Hello everyone!

Thank you for your patience - it took us some time to find the appropriate contact on Bing side. Pasting here the latest we have from the Bing Imagery team:

Thank you to everyone for the feedback regarding the resolution changes in iD editor. We are currently switching data sources so there may be some temporary degradation in resolution while we work out the integration with the new data source. A majority of the data has been updated to the new data source, but there are still some regions in which the data has not been fully implemented, including parts of the UK. The complete integration should be completed in the upcoming months.

In addition, we are in the process of requesting all other endpoints using the API to update to the current version of the API. Communications to affected entities will be sent out shortly.

[RedAuburn]

This seems to be fixed now! (at least in London)

[RedAuburn]

this has been an issue for way too long, maybe iD could temporarily use the workaround url until the correct ones are fixed?

[tyrasd]

@endim8 Well, the URL used by iD is the correct one. It's unfortunate that Microsoft has (AFAIK) not yet communicated that to the maintainers of other OSM editors such as JOSM. But there's just nothing I can do here: switching back to the old tile URLs would be quite similar to just adding google maps as a background to iD, ignoring their terms of use and licensing agreements.

[SK53]

I don't think this should be closed whilst Bing imagery on the provided link is over 10 years old.

The correct response is probably to announce that Bing imagery will no longer be the default, and replacing it, with say ESRI imagery. I suspect we may need to consider retiring Bing imagery completely.

[simonpoole]

@SK53 I in principle agree, the question is if it should just be retired here or if we should remove it pre-emptively from other editors, as has been mentioned Microsoft has not reached out to any of the editor devs as far as I know (including myself).

[nbaksalyar]

I suspect we may need to consider retiring Bing imagery completely

I wonder if the changes made using this imagery need to be rolled back - I've been also using the custom URL from RapiD/JOSM. Considering that a lot of people use these editors and Bing's imagery too, what should be done about that? This seems to be a complicated licencing issue.

[timothywashere]

What we need is a definitive statement from someone at Microsoft/Bing about why it has changed and if it is permanent. Someone who understands what the issue actually is

Personally I've stopped adding houses to my area, the alternative images are just too poor to see where individual buildings begin and end

[simonpoole]

What we need is a definitive statement from someone at Microsoft/Bing about why it has changed and if it is permanent. Someone who understands what the issue actually is

Unluckily this the kind of thing which you can't force . Pinging the OSMF board and the EWG (which IMHO should be managing this to start with) @grischard @tordanik @pnorman potentially fixable via Microsofts member on the advisory board.

[SK53]

@simonpoole @grischard Possibly should be raised via the Advisory Board representative from Microsoft. At this stage I think we need as much clarity as possible, but large company's seem to be structured in such a way that makes this quite hard. I note that the Bing Satellite layer on the National Library of Scotland is rate limited, which I never noticed in the past.

[boothym]

I don't think this should be closed whilst Bing imagery on the provided link is over 10 years old.

The correct response is probably to announce that Bing imagery will no longer be the default, and replacing it, with say ESRI imagery. I suspect we may need to consider retiring Bing imagery completely.

Recently noticed someone add buildings and features which were no longer there, and someone else remove a construction area because it looked like a field back in 2012 on Bing. It can't be good to have editors (in this case one with hundreds of edits and the other a new editor) having such old imagery as the default without letting them know what the situation is.

Personally I've stopped adding houses to my area, the alternative images are just too poor to see where individual buildings begin and end

As mentioned earlier in this issue, the 1.1.9 version (rather than the 2.0.0-alpha3.2 version) of the RapiD editor at https://mapwith.ai/rapid still has the high quality Bing imagery, and as a bonus it also has Microsoft building shapes which you can add to save time (assuming they are accurate/aligned in your area).

[AlexT25]

I've just noticed that the default Bing imagery (in my area of the UK at least) has changed from low quality, very out of date, imagery to much higher definition up-to-date imagery. So it appears this may have been fixed.

[RedAuburn]

i'm getting the same (for real this time)! glad it's been fixed 😄

[nc011]

I don't think it's fixed everywhere in the UK unfortunately. Was editing here: https://www.openstreetmap.org/edit#map=20/50.33137/-4.51779 and Bing is still just Esri (beta) but offset (you can tell as the boats in the harbour are the same). RapiD shows higher res Bing imagery.

[timothywashere]

I don't think it's fixed everywhere in the UK unfortunately. Was editing here: https://www.openstreetmap.org/edit#map=20/50.33137/-4.51779 and Bing is still just Esri (beta) but offset (you can tell as the boats in the harbour are the same). RapiD shows higher res Bing imagery.

Yes, Everywhere I've checked in Devon seems to be high res but parts of Cornwall still stuck on the old low res unfortunately.

[tsmock]

Do we know which layer Microsoft/Bing wants us to use? #9153 (comment) seems to indicate AerialOSM, but AFAIK Microsoft never reached out to other projects (in my case, JOSM).

Furthermore, it is an undocumented layer (see Bing's documentation on get-imagery-metadata), and I generally don't like using undocumented features. Especially when the provider has not reached out and told us we can use it.

[simonpoole]

@tsmock I suspect that if anything getting an answer out of Microsoft is even more difficult than a year ago. As you note they never reached out to other projects, and it isn't really even clear what we should be using, if anything at all, in the 1st place.

I don't believe the board or the EWG ever tried to clear things up either.

[tsmock]

I hate it when we are in limbo with an external provider. I don't want there to be a story whereby Bing had to shut down service to OSM editors because we were accidentally using the wrong endpoint.

[osm-spiregrain]

@tsmock I suspect that if anything getting an answer out of Microsoft is even more difficult than a year ago.

Perhaps following this news: - https://blog.openstreetmap.org/2023/10/25/microsoft-pledges-150k-to-support-openstreetmap/

... there is now a friendly contact between MSFT and OSMF which could be used to unstick this and set our use of Bing images (aerial and streetside) on a firmer foundation?

[mikelmaron]

they are already aware and discussing with the appropriate team internal to microsoft, and plan to comment on ticket soon.

[ninoslavp]

Thank you for your patience!
It took some time to convey all the feedback and aggregate the inputs across teams involved. Sharing what we have:

Due to increased usage of Bing Maps imagery APIs, we decided to separate the free usage of the API (for OSM editors) from the paid usage of the API. Due to the separation and some variation in the data sources between the two endpoints, in some circumstances, lower quality images were served through the OSM endpoint than through the non-OSM endpoint. Our imagery team has worked on addressing the lower quality imagery to ensure that both endpoints are consistently serving high quality imagery. If you are still aware of any gaps in quality, please let us know, so we can follow up and improve service quality for mappers across the World.

I would be happy to proxy any outstanding image quality concerns, or you can use osm@microsoft.com to get better traction.

[simonpoole]

@ninoslavp thanks for the response. While this does explain the motivation, it doesn't address the concerns and questions from developers working on projects other than iD. See for example #9153 (comment)

[timothywashere]

If anyone is contacting Microsoft it seems the license terms that osm uses for bing Streetside doesn't allow for objects to be added to OSM based solely on Streetside imagery, that it can only be used to confirm details of objects that are mapped some other way. It would be nice if this restriction was relaxed so i could add telephones, defibrillators and postboxes i can see on Streetside alone.

[bhousel]

If anyone is contacting Microsoft it seems the license terms that osm uses for bing Streetside doesn't allow for objects to be added to OSM based solely on Streetside imagery, that it can only be used to confirm details of objects that are mapped some other way. It would be nice if this restriction was relaxed so i could add telephones, defibrillators and postboxes i can see on Streetside alone.

This was already clarified when they originally submitted the integration PR 👍
see #5050

[ninoslavp]

@ninoslavp thanks for the response. While this does explain the motivation, it doesn't address the concerns and questions from developers working on projects other than iD. See for example #9153 (comment)

I think the AerialOSM should be OK to use, but there are actually multiple new endpoints that also brings some differences in styles and pre-renders.
To be on safe side, I will assign someone from my team to look deeper into most popular OSM editors and propose the change in the following 2-3 weeks.

[osm-spiregrain]

I will assign someone from my team to look deeper into most popular OSM editors and propose the change in the following 2-3 weeks.

Some fairly recent work to identify the most popular editors is here: https://wiki.openstreetmap.org/wiki/Editor_usage_stats . Not all of these editors present Bing Streetside or aerial imagery at all, though.

[nc011]

Thank you for your patience! It took some time to convey all the feedback and aggregate the inputs across teams involved. Sharing what we have:

Due to increased usage of Bing Maps imagery APIs, we decided to separate the free usage of the API (for OSM editors) from the paid usage of the API. Due to the separation and some variation in the data sources between the two endpoints, in some circumstances, lower quality images were served through the OSM endpoint than through the non-OSM endpoint. Our imagery team has worked on addressing the lower quality imagery to ensure that both endpoints are consistently serving high quality imagery. If you are still aware of any gaps in quality, please let us know, so we can follow up and improve service quality for mappers across the World.

I would be happy to proxy any outstanding image quality concerns, or you can use osm@microsoft.com to get better traction.

As far as I can tell Cornwall (county in the UK) is still low res Esri at low zoom levels. But would appreciate someone else confirming just in case it's an issue my end!

[mbrzakovic]

Hi all,

The guidance to all OSM editors for using Bing Maps imagery APIs is the following:

Follow the call pattern described in this documentation:

1. During the startup of an app (or at session start) call imagery metadata and provide AerialOSM for imagerySet parameter. Optionally, you can also use include=ImageryProviders to get imagery providers in response as well,
   and uriScheme=https to get https template url (as described here)

2. For all tile-specific calls during the session/app lifecycle, use the template from retrieved response of 1) to create full image URL for each tile.

iD has the correct implementation for this pattern - code (part: rendererBackgroundSource.Bing), feel free to use it as an example

[ninoslavp]

As far as I can tell Cornwall (county in the UK) is still low res Esri at low zoom levels. But would appreciate someone else confirming just in case it's an issue my end!

@Caseyb87, thanks for pointing out. I have reached out to the imagery team and there was indeed glitch with southwest corner. It should be fixed now. If you are still experiencing issues, just drop me a note.