Skip to content

Home

Jump to bottom
Phillweston edited this page Mar 27, 2022 · 4 revisions

Welcome to the Rosettadrone 2 wiki!

**Finally, we got some good resources helping out, the video issue will be checked in ASAP, as well as other issues with GOTO, etc. **

To FLY YOU MUST DISABLE THE SAFETY LOCK IN THE UPPER RIGHT CORNER::::

I will start writing down what I do, by a stroke of luck I got the GStreamer latency down to almost nothing. See this video on YouTube as a benchmark for future work.

https://www.youtube.com/watch?v=HAC5OpQQpDI&feature=youtu.be

Testing: Be aware that you can tap 5 times on the drone image at the boot screen to bypass the need for connecting a drone (for GUI work only)

Simulator: Also, you can tap the simulator to test you're flying without actually fly, but then you need the actual Drone connected.

Compatibility: Still no video to QGC for Mavic Air, but other than that is also works fine.

Buttons: C1, C2, and C3 buttons on the DJI controller is mapped to CH 5, 6, and 7 on the RC to QGC or MavProxy/DroneKit CH 8 = the AI button, this allows the user to start additional functionality on the PC without having to move the focus from the controller. In Dronekit you can subscribe to these buttons to activate functions...

RosettaDrone 2 is now updated to DJI SDK 4.16.1...

3. Des. 2020: Main is merged with the work branch, thanks for all the help.

Rosetta Drone 2 tested on DJI Air, Mavic 2, Mavic 2 Enterprise Advanced, and Matrice 210 V2 with X7 camera series and supports Android 5.1 and newer, with DJI SDK 4.16.1 and Androidx lib

=======

The project works with QGroundcontrol.

No Video on the Mavic-Air at the moment, but working on it...

Please create issues and create PR on this fork.

Image of Opening screen Opening screen requires DJI login the first time...

This version has implemented the new video stream method in DJI SDK that requires quite a few modifications. It also includes all the latest libraries and Androidx. In the unstable code, there is also the DJI simulator, to enable testing in the lab. Android Studio is now updated to 2020.3.1.

Rosetta Drone is a Mavlink wrapper for the DJI SDK, which allows users to fly DJI drones using Mavlink-speaking ground control stations. In theory, it should work with any Mavlink GCS, but all testing so far has been done with QGroundControl.

*** IMPORTANT SAFETY NOTE *** Due to imperfect translation between DJI and Mavlink, props may begin spinning at unexpected times. Always treat props as if they are live. Use Rosetta Drone's "safety" feature, which should prevent the drone from acknowledging unexpected GCS arm or takeoff commands.

The user assumes all responsibility for the prevention of harm or damage. This is an evolving, experimental app. See "Known issues" below before use.

Image of Opening screen Main screen, with live video on Mavic AIR (from my office)...

Image of Opening screen Main screen on QGroundControl with live video and telemetry... (from inside my office)

Image of Opening screen Rosetta Drone 2 on the big screen from DJI Smart Controller

Image of Opening screen Rosetta Drone 2 on the DJI Smart Controller

Latency video: Video (https://youtu.be/HAC5OpQQpDI "Click to play in Youtube.com")

Features

  • Report telemetry in QGC like position, attitude, relative altitude, heading, and battery remaining
  • Command Return-to-Launch from QGC
  • View drone video feeds in QGC or forwards RTP to an IP address of your choice (currently Mavic Pro 2 only)
  • Create and fly waypoint missions
  • Fly by joystick and QGS
  • Fly from Python in DroneKit
  • Use Mavproxy to connect both QGC and DroneKit at the same time
  • Use Gstreamer/OpenCV/FFMPEG and DroneKit to create complex AI functions

Usage

Setup (QGroundControl)

  1. Connect your Android phone to a DJI transmitter and power on your DJI drone.

  2. Start Rosetta Drone. The DJI light in the top-right will turn green if the app is successfully communicating with your drone.

  3. If you wish to use QGroundControl on an external device, click the Gear icon to access Settings, check Use GCS on an external device, then specify an IP address.

  4. Start QGroundControl. A telemetry connection should be immediately established, and the GCS light in Rosetta Drone will turn green. Note that if you are using QGroundControl on the same device as Rosetta drone, the GCS light may not turn green if QGC is in the background.

  5. To start video:

    a. Click the "Q" icon in the top-left corner of QGC

    b. Under Video change Video Source to UDP Video Stream.

    c. Change UDP Port to 5600.

  6. Takeoff is recommended using the RC transmitter. To arm or takeoff from the GCS, click the SAFETY ENABLED button. It will turn green and say READY TO FLY. Then use the QGroundControl Takeoff or Start Mission function.

  7. After the flight, ensure that safety is enabled before approaching props.

  8. Support is added for Pan and Tilt of camera gimbal by using RC channel 8 and 9.

Add this to you're Dronekit python script:

def set_servo(servo,val): msg = vehicle.message_factory.command_long_encode( 0, # time_boot_ms (not used) 0, # target system, target component mavutil.mavlink.MAV_CMD_DO_SET_SERVO, 0, servo, # RC channel... 1500+(val*5.5), # RC value 0, 0, 0, 0, 0) vehicle.send_mavlink(msg) vehicle.flush()

And then use: set_servo(8,-45.0) to set the gimbal...

  1. Support for joystick is added and tested with QGroundcontrol, using the now built-in simulator.

  2. By taping on the drone icon 5 times, you enable test mode and can open the software with no drone connected. For GUI work.

Building from source

  1. Clone or download the repository.

  2. In Android Studio, select File->New->Import Project and navigate to the downloaded folder.

  3. Sign up for the DJI Developer Program at https://developer.dji.com/mobile-sdk/ and create an Application key. The package name should be sq.rogue.rosettadrone.

  4. Create a new file called keys.xml in the /values folder, and insert the following:

    <?xml version="1.0" encoding="utf-8"?>
<resources>
    <string name="dji_key">INSERT KEY HERE</string>
</resources>

  5. Run Build->Make Project

DJI-Mavlink Translation Notes

Anyone who speaks multiple languages knows that translations are rarely perfect. The same is the case here.

  • The ARM button in QGC does not work, by design. Sending a "Takeoff" or "Start Mission" command from QGC will arm the motors and takeoff.

  • DJI reports heading in True, which Rosetta drone 2 passes along in vfr_hud.hdg. The mavlink protocol does not specify magnetic or true.

  • DJI and Mavlink use different scales to characterize GPS accuracy. DJI also does not report hdop or vdop.

  • Radio signal strength is not yet implemented

  • Rosetta drone 2 reports groundspeed and airspeed as the same and does no wind correction

  • There is currently no way to turn the video camera on or off in QGC, but DJI controllers will still work fine, Mavlink support for this is now added.

  • The only implemented waypoint actions include delaying at a waypoint, taking a photo, or changing the gimbal pitch

  • Battery % is now shown in the GUI on the main screen.

  • Battery Warnings for both Drone and Controller at 20% 10% and 5%. No sound at the moment.

  • Controller stick positions and C1, C2, and C3 switch states are now reported as RC RAW messages over Mavlink

  • A working DJI simulator is added for testing and training. The simulator is Hardware In the Loop (HIL), and this requires both a controller and a drone connected.

  • A test mode is added by hitting 5 times on the drone icon. This will enable GUI testing only.

  • The message on the boot/login screen can now be modified in "settings/drone configuration/App Name", to make the app more private.

  • The switch C3 is now both reported as switch 3 and also perform an auto takeoff if the system is armed. This to allow flying from Rosetta Drone 2, might be reverted.

Known Issues for Users

  • If you change the UDP port while QGC is running, you may need to restart QGC.

  • If you change the video UDP port while QGC is running, you may need to restart Rosetta drone 2

  • Sometimes the Rosetta Drone 2 stops and restarts, work are in progress to eliminate this. Seems to be related to the Mavlink layer.

  • If you takeoff using the QGC, and then set a point and hit goto, the Rosetta drone 2 enters an error state. Work is in progress.

  • If the APP is running in the background, the QGC video screen may stop playing. Therefore, it is recommended to use an external device to play video footage with QGC.

  • In order to ensure the quality of the connection between QGC and app, it is recommended to open QGC first and then start the APP.

TODO

  • Fix goto position.

  • Fix mission plan flight.

  • Add more actions to the mission waypoint

  • Test and prepare official release, add to play store.

Known Issues for Developers

RosettaDrone 2 uses MAVLink code generated by the MAVLink code generator, using the ArduPilot dialect. The Java code generator contains errors (see issues #805 and #806), and the code required manual tweaking after generation. This means that simply dropping in updated auto-generated files will likely produce errors. At the moment this seems only to apply for setting and getting double that is missing. It is quite easy to copy the float function and modify it to double. The only usage for this is for ROV wheel distance, so someone should fix that in Mavlink.

The use of an ArduPilot dialect over PX4 is not intended to be a statement of preference. The author believes strongly in the importance of maintaining maximum compatibility across both projects.

Acknowledgements

Rosetta Drone was brought to you by developers from Rogue Squadron, a UAS/C-UAS red team at the Defense Innovation Unit Experimental.

The Rosetta Drone 2 version was brought to you by the Drone department at 9Tek AS, Norway.

Additional features was designed and verified by Phillweston, Yuwoooo at XiDian University.

Icons

Icons made by Pixel Perfect & Freepik at flaticon

Project icons made by Phillweston at XiDian University.

Licensing

Rosetta Drone 2 is licensed under BSD3. For particularities related to U.S. federal government code release, please read Intent.MD. For more information, visit code.mil or code.gov.

Rosetta Drone 2 uses a modified version of DJI's Android Video Stream Decoding Sample, which is released under the MIT License.

Rosetta Drone 2 uses a java version of OpenCV 4.5.4, which is released under the Apache 2 License.

Video RTP packing uses code modified from the libstreamer library, licensed under Apache 2.

Clone this wiki locally