Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pixhawk 6c mini external compass disappearing #25946

Closed
ashwanisihag opened this issue Jan 10, 2024 · 35 comments · Fixed by #26929
Closed

Pixhawk 6c mini external compass disappearing #25946

ashwanisihag opened this issue Jan 10, 2024 · 35 comments · Fixed by #26929

Comments

@ashwanisihag
Copy link

When we use pixhawk 6c mini with m9 or m10 the external compass disappeard after few flights. Its been reported in almost every system that external compass is missing. Internal is always detected but external disappears. May be because they are same Ic.

We have checked everything and all seems normal when we reinstall firmware. Its not a loose connection problem.
@boba1234567890
Copy link

Hi I've also been having this problem but with my Pixhawk reinstalling the Sofware (ArduCopter v4.4.4) is not solving anything.
even downgrading to (v4.4.3) it still doesn't work.

@ashwanisihag
Copy link
Author

Here is Full description to duplicate the issue and solution:

Pixhawk 6C Mini and M10 or M9 both has same compass IST8310 Compass. So after few flights external will be not available or not detected or some thing goes wrong when we have same ic as external and internal as compass.

Solution: Not reliable but restarts works sometime but not always.

@rmackay9 rmackay9 mentioned this issue Feb 19, 2024
Open
92 tasks
@rmackay9
Copy link
Contributor

@ashwanisihag, @boba1234567890,

Can you tell us exactly which compass module you are using? Is it part of a GPS unit? Perhaps provide a picture if possible. Also if you have an onboard log that would be great. You should be able to create a log even if the autopilot is not armed if you first set LOG_DISARMED = 1.

@rmackay9
Copy link
Contributor

@ashwanisihag @boba1234567890,

Sorry to pester but could you clarify which compass module is being used?

@rmackay9
Copy link
Contributor

Hi, no feedback so I think I'll close this issue now. If we do hear back from @ashwanisihag or @boba1234567890 then I'm very happy to re-open.

@ashwanisihag
Copy link
Author

ashwanisihag commented Mar 17, 2024 •
edited
Loading

pix
Hi Randy

Sorry I missed your message was bit busy with army..thanks for replying...Please find the photo attached above of the hardware we are using. Its a big problem now as we have bought 1000 FC units and all have the same issue.

By the way its pixawk 6c mini and M9 or M10 gps units from holybro. After few restarts it works fine but when on training with army external compass sometimes disappears on ground and its frustrating.

Because the external compass disappers it says "Compass inconsistent"

If this is fixed then I will test 4.5.0 today and will report you asap because it comes rendomly and not so easy to duplicate.

Regards

@ashwanisihag
Copy link
Author

ashwanisihag commented Mar 17, 2024 •
edited
Loading

@ashwanisihag, @boba1234567890,

Can you tell us exactly which compass module you are using? Is it part of a GPS unit? Perhaps provide a picture if possible. Also if you have an onboard log that would be great. You should be able to create a log even if the autopilot is not armed if you first set LOG_DISARMED = 1.

HI compass is IST8310 in M9N or M10N
Pixhawk 6c Mini has also the same compass onboard

@ashwanisihag
Copy link
Author

Just tested

Same issue in 4.5 beta
Screenshot 2024-03-17 161625
1 01-01-1970 05-30-00.zip

@ashwanisihag
Copy link
Author

ashwanisihag commented Mar 17, 2024 •
edited
Loading

Is this the issue?

"The IST8310 is more complicated to auto detect because it has 4 different I2C slave addresses."
https://intofpv.com/t-discover-compass-ist8310-in-betaflight
Although its related to betaflight code

@rmackay9 rmackay9 reopened this Mar 17, 2024
@rmackay9
Copy link
Contributor

Hi @ashwanisihag,

Thanks for the feedback and links, I've re-opened the issue and hopefully we can reproduce it and/or just fix it.

@tridge
Copy link
Contributor

tridge commented Mar 19, 2024

@ashwanisihag we do look for 4 different i2c addresses for probing:
https://github.com/ArduPilot/ardupilot/blob/Plane-4.5/libraries/AP_Compass/AP_Compass.cpp#L1232
if you have spare boards that reproduce this issue reliably you could send one to me
otherwise we may need Saleae pro traces of the i2c bus

@tridge
Copy link
Contributor

tridge commented Mar 19, 2024

@ashwanisihag feel free to ping me on discord too

@ashwanisihag
Copy link
Author

ashwanisihag commented Mar 20, 2024 •
edited
Loading

@ashwanisihag we do look for 4 different i2c addresses for probing: https://github.com/ArduPilot/ardupilot/blob/Plane-4.5/libraries/AP_Compass/AP_Compass.cpp#L1232 if you have spare boards that reproduce this issue reliably you could send one to me otherwise we may need Saleae pro traces of the i2c bus

Yes, I can send you one set. Please guide me how to send and at what address.

@ashwanisihag
Copy link
Author

@ashwanisihag feel free to ping me on discord too

Sure i will try

@ashwanisihag
Copy link
Author

@ashwanisihag we do look for 4 different i2c addresses for probing: https://github.com/ArduPilot/ardupilot/blob/Plane-4.5/libraries/AP_Compass/AP_Compass.cpp#L1232 if you have spare boards that reproduce this issue reliably you could send one to me otherwise we may need Saleae pro traces of the i2c bus

I have seen this code and tried to figure out but was not abe to figure it out.

@rmackay9
Copy link
Contributor

@ashwanisihag,

I'd like to get to the bottom of this if possible. Could you ping Andrew Tridgell or me on Discord? I think maybe your Discord id is different from your GitHub id?

@rmackay9
Copy link
Contributor

rmackay9 commented Apr 1, 2024

No updates for a week so I'll close this issue again. We will see responses however.

@rmackay9 rmackay9 closed this as completed Apr 1, 2024
@ashwanisihag
Copy link
Author

No updates for a week so I'll close this issue again. We will see responses however.

I will create Discord id and will connect. Thanks

@rmackay9
Copy link
Contributor

I've received another ping that this issue is real.

@rmackay9 rmackay9 reopened this Apr 16, 2024
@Craig-1965
Copy link

Hi all.

I have had the same problem with the PH6C Mini

First issue was in a field test, I unfortunately had the internal compass temporarily disabled. The drone had been flying perfectly and was well tuned. All of a sudden it turned 90 degrees and flew off at a speed in excess of 120km/h, never to be seen again.

The next drone I had to disarm and violently crash a flyaway mid flight due to the External compass disappearing an the Satellite connection degrading drastically (simultaneously) it dropped 60% of Connected satellites.

Product support from Holybro was poor and they attempted to place the blame cack on my setup. To add context we have over 250 units in flight using the PH4 Mini and Pix32 from Holybro and no other such issues.

I have tested the setup ad nauseum using the HolyBro M8N, M9N and newest M10N and had intermittent faults on all. Just powering the drone up and letting it stand for an hour,, powered up and along the way the External Compass Disappears and connected satellites will drop fom 18-21 down to 9-11.

@Craig-1965
Copy link

With continued communications with Holybro I received the following Email 11 March 2024 (and nothing since then nothing)

**Hello,

We have found that the lost external compass on a Pixhawk 6c running Ardupilot connected with a M9N GPS is caused by the Ardupilot firmware.
We need some time to get it fixed.
Now please give it Px4 instead which is free of such an issue.
Will keep you updated.

Thanks!

Holybro**

@vincentpoont2
Copy link
Contributor

vincentpoont2 commented Apr 16, 2024 •
edited
Loading

With continued communications with Holybro I received the following Email 11 March 2024 (and nothing since then nothing)

**Hello,

We have found that the lost external compass on a Pixhawk 6c running Ardupilot connected with a M9N GPS is caused by the Ardupilot firmware. We need some time to get it fixed. Now please give it Px4 instead which is free of such an issue. Will keep you updated.

Thanks!

Holybro**

This issue indeed only occur with Ardupilot, but sorry that our rep tell you to "give it Px4 instead which is free of such an issue".

We are working with @rmackay9 @tridge to fix this issue in Ardupilot. The hardware are already on the way to tridge for troubleshooting, and we will provide update here when we have a solution.

Vince
Holybro

@Craig-1965
Copy link

Doesn't do anything for the units lost and other damages or the approach followed by the Support staff.

I'll keep the other units out of the air until addressed.

What doesn't make sense and as discussed with he support staff. Without a UBEC powering the Servo rail it remails a lot more stable. I have one unit that I personally fly and I have around 22 hours of flying on it without incident. It just so happens that this Servo rail is powered by a 5A UBEC from Tarot and if I change the UBEC to a different brand the instability returns.

Doesn't make sense I know but this is live experience (may or not be of value) and this unit is one of the best drones I have in use at the moment.

@rmackay9
Copy link
Contributor

@Craig-1965,

If you have a log of the compass failing/disappearing in flight we'd really like to see it. If that can be done without flying it all the better, maybe set LOG_DISARMED = 1.

@Craig-1965
Copy link

It will be IMPOSSIBLE to get the Logs for the ones where it happened in Flight.....

Those 2 Drones, Cameras, Payloads etc are still GONE/LOST/MISSING. They did a sudden direction change and both flew off at 120km/h+ Until the battery ran out and were never seen again. The one went into Alt Hold about 3.5km from home and I managed to fly it back on the screen for about 100m and then it shot away at 90 degrees to flight path and I lost comms at 6.1km over a residential area.... Luckily only travelling at 18m/s when it went down. I haven't seen reports of injury or property damage, but I am sure there was.

I'll just have to get the SD Card from the last used 6C Mini. I will try the LOG_DISARMED = 1 and send the recorded log.

I'm just out of country right now so will probably only get it done at the earliest next week.

@tridge
Copy link
Contributor

tridge commented Apr 30, 2024

@ashwanisihag @Craig-1965 I have the hardware now, sent by @vincentpoont2 - what I need is a way to reproduce the issue. So far I've been testing on USB power, with numerous soft reboots to see if it ever comes up without the compass. I also tried having it sit for a long time to see if the compass would disappear. No luck so far reproducing the issue.
do either of you have suggestions on approached to triggering the issue?

@Craig-1965
Copy link

Craig-1965 commented Apr 30, 2024 via email

@tridge tridge mentioned this issue Apr 30, 2024
Merged
@tridge
Copy link
Contributor

tridge commented Apr 30, 2024

@Craig-1965 @ashwanisihag I have now managed to reproduce the issue and I've opened a PR that I think will fix it:
#26929
I've put builds of plane and copter 4.5.1 with this fix here:
http://uav.tridgell.net/tmp/IST8310/
Please test!

@Craig-1965
Copy link

Craig-1965 commented Apr 30, 2024 via email

@rmackay9
Copy link
Contributor

@Craig-1965,

I've checked Tridge's link and there does not appear to be any password required to download from the site. Maybe you could try again with a different browser?

image

@tridge
Copy link
Contributor

tridge commented Apr 30, 2024

@Craig-1965 the problem was likely your browser auto-converting to https and my servers SSL records being corrupt
please try this now:
https://uav.tridgell.net/tmp/IST8310/
I have also put them here:
https://drive.google.com/drive/folders/12VJm_4b4449iJcfqIM1okFV8p19D-_WU?usp=sharing

@Craig-1965
Copy link

Craig-1965 commented May 1, 2024 via email

@tridge
Copy link
Contributor

tridge commented May 1, 2024

@Craig-1965 if your testing goes well we will include it in the 4.5.3 release. We did not include in todays 4.5.2-beta as we don't yet have confirmation it fixes your issue. Also note that @vincentpoont2 from Holybro has indicated a hardware fix may be coming

@ashwanisihag
Copy link
Author

ashwanisihag commented Dec 15, 2024 •
edited
Loading

It seems like issue has been fixed in latest firmware's ( may be after 4.5.3 release). Am i right andrew? Because i have not seen this issue again after firmware update.

@rmackay9
Copy link
Contributor

Hi @ashwanisihag,

Yes, the fix from Tridge was included in the next release, 4.5.3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

AP_Compass: workaround hardware bug in IST8310 whoami tridge/ardupilot
6 participants
@tridge @rmackay9 @ashwanisihag @vincentpoont2 @Craig-1965 @boba1234567890