-
Notifications
You must be signed in to change notification settings - Fork 207
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
I2C regression, last working commit is 43998c82a7e88df284b7aa30221b2d0d21b2b86a #261
Comments
Can you confirm that between working and non-working systems |
Or to word it better, can you confirm that the same version of the firmware can be found in the last working and first non-working system? |
Last Raspbian stable, I2C NOT working :
OLDEST version with I2C NOT WORKING :
LAST version with I2C working :
|
Thanks. I'd dearly love it to not be my "untimely DONE signal" patch... If you don't mind a bit of experimentation, starting with the last working commit (43998c8), do a |
Hi The next firmware version with the old kernel work, it's a success for you but please check if I understand all (my english is aproximative) : From the last-working & complete version on one robot;
It's good ? |
Or just ask for more tests... |
Thank you for doing that - you've narrowed the problem down to a kernel change in the 5.10.14 update. As you thought, I'm tempted to revert that one commit I'm suspicious about, but I want to discuss with colleagues if there is a better way to distribute trial releases rather than change it for everybody. |
What is the IMU device(s) you are using? It would be useful to get some for testing. Also, do you a very simple test script that demonstrates the problem? |
Now I try to make the bug visible out of our Vigibot.com software stack : On Vigibot we read the I2C bus from two simultaneous process :
On Vigibot we use RTIMULib official binaries from Raspbian repo.
And easy to run
The process need CTRL + C to exit. I found strings of the problem inside C++ code "I2C read error from 104, 116 - Failed to read fifo data"
Now I need to return to a bad Kernel and reproduce the problem with the RTIMULibDrive process. |
You need a mpu-9250 breakout board : https://www.amazon.fr/gp/product/B07HMQZ7N9 Don't buy the blue PCB version without pressure sensor : |
Win ! You can use the compiled binary to get the problem, The error is more frequent with the Node.js client I2C slow pooling for battery monitoring. + the binary for testing :
I stop the vigibot client
But error still here, but very rare need a long run :
The raw trace from our C++ process received on the website is :
or
|
I remembrer we use 400K I2C bus rate :
I just tested @ 100K and it's harder to get the problem ! we must run @ 400K ! |
Thanks - I have one on order, but it might take a month to arrive. |
Important you need 400KHz I2C clock and 1KHz IMU sample rate to get multiples I2C errors every minutes.
Few minute testing on the last Master (kernel + firmware) :
Also I can give you a root SSH access to a fresh robot with FTTH internet access if you want to try things before the package to arrive (you can crash it no problem, they are from my automatic integration). |
That's a good idea. Email me - phil@raspberrypi.com - with the details. My plan would be to push a trial kernel to it to confirm that it solves the problem before I make the change public. |
Now you can ssh to my domain with the email port + password. It run on the last working version, all is ready to work or fail depend on the Kernel version. |
Thanks - expect a visit soon. |
You can reboot as you want... I keep open access for multiples days if
you need....
---
Pascal
https://www.vigibot.com
https://www.serveurperso.com
Le 17/03/2021 15:44, Phil Elwell a écrit :
Thanks - expect a visit soon.
--
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub [1], or unsubscribe [2].
|
You can look https://www.youtube.com/watch?v=NmrKH4Qx6io how Vigibot perform on a smartphone :
Everything @ 30FPS and 0.1/0.2 second end to end / 4G to 4G latency. |
Your target Pi is now running the latest kernel and firmware but with the suspect commit reverted, and I've not seen any I2C errors in a few minutes - with the current rpi-update kernel it saw errors after about 10 seconds. On the basis of that I'll push the reversion to our current branches, then revisit the patch once the MPU9250 arrives. |
If you want to try the kernel on other Pi 3s, download install.tgz from here (https://drive.google.com/file/d/1ItRLCyvgba8cQeU6qgtloxu_xgDsG-df/view?usp=sharing), or use:
directly on the command line. Install with:
(You will need to use sudo unless you run as root). |
I confirm you the complete system work on the kernel (vigibot.com robot name is Lidar) If you can give me a procedure I can deploy on the 10 testing robots |
N.B. The test kernel is only for a Pi 3B, 3B+ or 3A+ or 2B. |
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
LOL I see the answer few ms. before I clicked the "Comment" button. you can check my executing your procedure here :
|
Yes - that looks correct. |
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
commit ffb6947602d01b4dd60cc537f48e550332242b30 from https://github.com/raspberrypi/linux.git rpi-5.12.y This reverts commit 431ac1d21b5e842c5a564bc0bab31d7dc11611f5. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com> Signed-off-by: Meng Li <Meng.Li@windriver.com>
kernel: Make rpi poe fan less noisy in cool environments See: raspberrypi/linux#4222 kernel: overlays: Add pcie-32bit-dma overlay See: raspberrypi/linux#4216 kernel: ARM: dts: bcm2711: Add aliases for additional SPIs kernel: Revert i2c: bcm2835: Handle untimely DONE signal See: Hexxeh/rpi-firmware#261
kernel: Make rpi poe fan less noisy in cool environments See: raspberrypi/linux#4222 kernel: overlays: Add pcie-32bit-dma overlay See: raspberrypi/linux#4216 kernel: ARM: dts: bcm2711: Add aliases for additional SPIs kernel: Revert i2c: bcm2835: Handle untimely DONE signal See: #261
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
commit 13c43880f2ee6db7d26949cb6a3e8db1a4b76736 from https://github.com/raspberrypi/linux.git rpi-5.10.y This reverts commit 431ac1d21b5e842c5a564bc0bab31d7dc11611f5. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com> Signed-off-by: Meng Li <Meng.Li@windriver.com>
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
My today testing :
The master branch firmware (and kernel) are OK ! |
Please close the issue, |
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
commit 26efbc6bfcffdfa389d8870adc95bc210bc2954b from https://github.com/raspberrypi/linux.git rpi-5.12.y This reverts commit 431ac1d21b5e842c5a564bc0bab31d7dc11611f5. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com> Signed-off-by: Meng Li <Meng.Li@windriver.com>
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
This reverts commit 431ac1d. Reverting this patch because it appears to have caused a regression when used with an MPU-9250 IMU. See: Hexxeh/rpi-firmware#261 Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Hello,
With last raspbian apt update I get an I2C kernel problem on every platform (all RPI version) with :
RTIMULib IMU reads become unreliable (random corruption and FIFO overflow) on all RPI robotics platform >= kernel 5.10.14
RTIMULib IMU reads work perfectly on all RPI robotics platform <= kernel 5.10.11
Last working commit is 43998c8 (kernel: spi: bcm2835: Workaround/fix for zero-length transfers) Now we use this firmware commit ID on our official system image.
I2C regression can be inside :
See: media: i2c: imx290: Support V4L2_CID_ANALOGUE_GAIN as well as V4L2_CI… raspberrypi/linux#4119
See: Issue with i2c-bcm2835 module with I2C devices? raspberrypi/linux#3064
The text was updated successfully, but these errors were encountered: