GPS Initialization and Projection

[JeremyBYU]

Hi,
I just have a 2 questions about GPS in AirSim and a final generic question.

Question 1

How to set the initial GPS location (I believe this is called home_point in the code) . For clarification this is my setup:

OS: Windows 10
Flight Controller: Simple Flight
UE4 Custom Map - Pawn Spawns automatically

It seems that it is set to some Microsoft building (47.6414680,-122.140167) by default. I see it hardcoded here and here.

I think the first link is the relevant code and its being automatically set. So I can change this code myself manually and rebuild, but I was wondering if there was a more elegant way. Like maybe a settings files configuration, or maybe a custom property on the a vehicle pawn that is set on the UE4 editor (maybe UE4 has a GPS property on an object, I didnt see it). Or maybe an API to set the Home position (I'm using python and I dont see it there).

Question 2

I am looking at the projection that you are using to convert between GPS and NED (planar coordinates: x,y,z). I think the code is here or here. The first looks a like a very simple projection, but was wondering how you would feel about supporting other projections. Right now my map in UE4 is projected in EPSG:3857 and all object can be geo-referenced this way. I was wondering how you would feel about using proj4 to do the conversion for you between the planar coordinates/GPS coordinates (instead of the two links I showed). I think the big concern would be speed and adding an extra dependency. However this would allow any research to just simply specify the projection (using the SRID like EPSG:3857 maybe in a setting file) and the projection is taken care of. Just wondering!

Question 3

So I am NOT using PX4 SIL, nor ground control. Can you explain the benefit of these items and how they fit in to the AirSim global picture. Is PX4 a replacement for simple_flight? Something I miss is the ability to subscribe to data events like the pawn position/orientation, instead of having to query it like the python API provides. I come from ROS/LCM background so this would be like a topic that I can subscribe to. Does PX4 SIL or Ground control provide that ability. Anyways thanks for all your help and the great product.
Thanks,

Jeremy

[sytelus]

For #1: which vehicle are you using (car / multirotor (simple_flight / PX4))? We are planning to add this in settings file. Right now we don't have any other way than changing code, however.

For #2: Right now everything is from UE4 to NED. The UE4 coordinates are all internal and all of the AirSim code as well as APIs expose only NED coordinates. I think it might be good to support other coordinate systems. Is that what you are talking about? Or are you thinking about how UE4 to NED conversion happens? The current code for conversion is simpler but should be fairly accurate over large area that matters for multirotors. Of course, for cars its different matter...

For #3: PX4 SIL allows you to fly drones using PX4 firmware without having hardware such as Pixhawk or Pixracer. However its difficult to setup and get working. This is why we designed simple_flight firmware. Currently, well, its really simple and only had PID controllers but later we would add state estimation as well. PX4 SIL doesn't enable event subscription model. We are planning to add ROS adapter later (current APIs don't support push model). One think PX4 can do is allow listening to MavLink messages. That's as close as you can get to listening to events. However you will need significant technical know how on how to do this and in the end you get to listen to MavLink messages that might be convertible to ROS messages. That should work in SIL or HIL mode. We have app called LogViewer that does this.

[JeremyBYU]

Thanks for the quick response!

1. I am using the default multirotor model with simple flight. No PX4. I think a settings file would be good and maybe also an api call. I saw this api call but I think its for PX4 only. I can use this if necessary, but I would prefer not to.

2. No I am not talking about the conversion between UE4 and NED coordinates. From what I saw its just a difference in scaling (cm to m, as well as an inversion in z). What I am talking about is the conversion from NED coordinates (x,y,z) defined in a plane (referring to the (x,y) part) to geographic spherical coordinates like GPS. See this link for a detailed explanation between spherical vs projected coordinate systems. Specifically you have code (which I linked to), that converts the NED coordinates to GPS coordinates using some form of a simple transformation. However, every transformation/projection has its drawbacks in how it accurately respects either area, distance, or direction. This is why there are so many different forms of GIS coordinates systems. I was wondering if you would be okay with allowing us (the researchers) to specify what that specific transformation will be used by using the proj4 library (defacto open source GIS library for this purpose). EDIT - As I think about it more I think I am not actually using NED in my Unreal World for all my objects. I am using a specific map projection to construct the world with and additional state variable for height. I guess my true request would be if we could use different coordinate systems (i.e. map projections) instead of NED, and of course the GPS transformation would follow. Basically we specify what that map projection is and then you could use proj4 to convert to GPS.

3. This was really good information to read. I will take a look at LogViewer and see how you are doing it. P.S. I might make a high level diagram on how data flows and the communication protocol between all the elements in the system.

P.S. The whole GPS thing is important for my research. I am able to automatically construct a geo-referenced virtual world in UE4 that matches the real world. For example I can then send GPS positions of real buildings in the world, and the multirotor will then fly to these high fidelity reconstucted buildings in UE4 (this is already working). The only issue is that the GPS positions sent back by AirSim will be wrong because they are using a totally different transformation (also the home position is off). I can convert it post receiving, but I will need to know the exact transformation you are using. I hope this makes more sense, and here is a video so that you can see what I am talking about. Thanks again for all your help!

[M-Kasem]

Sorry for the off-topic comment, @JeremyBYU can you please provide more details about "constructing a geo-referenced virtual world in UE4 that matches the real world", any link that demonstrates how to match the real word or a source of data for the geo-images are greatly appreciated, thanks a lot.

[JeremyBYU]

Hi Mohamed,
I used this plugin for blender that converts OSM files into 3D meshes of buildings, lands uses, etc. (which can later be imported into UE4). All plugins for blender are licensed under GPL so the source code is distributed as well so I could modify it to my pleasing. I added the ability to pick an arbitrary projection using pyproj and some other things necessary for UE4 (construction of UV maps). In addition I extended AirSims python API class to add geo-aware commands. Hope that helps! If you want to discuss more we should probably take this offline, you can reach me at jdcastaATumichDOTedu.

Just to clarify my desire for this issue, because I think my response to sytelus was pretty confusing:

1. I would like the possibility of researchers to specify different coordinate systems besides NED (like map projections).
2. I would like the ability to set the GPS home position

Right now I'm just not using the GPS 'sensor' from AirSim. I just get the NED coordinates and convert them to GPS coordinates myself.

Thanks!

Jeremy

[sytelus]

We are process to expose sensor data through API which will include GPS. I'll update this thread as soon as this will be ready.

[madratman]

#1920 sensor APIs were added here
The ability to set GPS home was added in AirSim quite some time back via settings.json
Hence, closing this.

The geo-referenced import in UE4 is pretty cool! Marking as something to be added to the docs.
@JeremyBYU, if that part of your work is public now, please get in touch.

[JeremyBYU]

@madratman ,
I never made this specific work public, but I have been meaning to write it up. Once I have time I can write up a blog post about all the tools necessary to create a georeferenced world in UE4 and have seamless mappings (for AirSim data and commands) between GPS -> Projected Coordinated System -> UE4 Coordinate System. In other words you tell Airsim to fly to the GPS position of the Empire state building and it actually flys in the UE4 to a simple recreated Empire state building.

However I have made a recent paper about using Unreal Engine to simulate Rooftop Landings (does not use the methods just discussed). You can find the paper here and short video here. In the process I made many useful tools:

• Randomly generate assets inside an UE4 world (used this on rooftop buildings)
• Visualize 3D point clouds in realtime in AirSIM using Vispy
• An AirSim collection module that can record data at specified vantage points with a provided config file (specify sensors and rates etc., sensors seem to be more reliably synchronized than the default AirSim recording method). This is meant more for recording data about the UE4 environment (camara, lidar).
• C++ UE4 plugin that will generate a classified point cloud of the environment. Imagine if you paid someone to fly an airplane over a city and classify each point.

I think these other tools I mentioned are actually quite useful and will release them soon. I will probably do a write up of the tools sometime in July and release them all.

How should I get in touch when more of this stuff is ready. Should I just e-mail you directly?

[JeremyBYU]

I finished writing the blog posts and putting together a code repository.

Blog Post 1
Blog Post 2

Repository for allowing AirSim to understand other projection systems: AirSimGeo