Possible incorrect ldc_fref_divider for Fly/Mellow boards

[gloomyandy]

Hi, the settings for the Fly/Mellow boards uses ldc_fref_divider = 1 which will result in a 40MHz fRef. This would seem to exceed the maximum fRef setting when in single-channel mode which is 35MHz. See Table 43 in the LDC1612 datasheet. Is this correct? Have you been able to test with Fly boards and confirmed that they operate ok?

Note I've no idea if this is actually a problem or not, I'm currently working on adding similar touch support to RepRapFirmware for the Fly boards and am curious if operating the boards with these settings works or not. Up until now I've been following the datasheet and have been using a fref divider of 2 to reduce the operating fRef to 20MHz, but if possible I'd like to operate the part at a higher frequency. I'm also curious how you have determined the settle time used for the Fly boards and why it is different to the values used for the other probes. One thing I've been struggling to understand is if the settle time is actually used when in single channel mode, most of the documentation only seems to mention it in a multi-channel context.

Thanks for any information you can provide!

[vvuk]

Yeah, I was actually curious about that one too. I haven't built my Fly toolhead yet (maybe this weekend), but there are a few people that are using the Fly successfully both with this version and a previous one where I used the same frequency as the btt eddy by accident.

I went off of Mellow's instructions where the only thing they change is the frequency. REG_CLOCKDIVIDERS still gets set to 1/1. I agree that it seems wrong; I was going to investigate once I had my Fly board set up.

I'm not sure there's a huge benefit to running at a significantly higher frequency; I think stability of that signal is more important. Cartographer runs at 24MHz, generated via gpio from the stm32; btt eddy at 12MHz. However Mellow may have determined that the chip is fine with Fref at 40MHz. Table 43 in the datasheet also talks about Fref <= 55 MHz in multi-channel conversion mode, which.. I don't know how you get that when all you have is Fin (capped at 40MHz) and a divider.

For settle time, I'm pretty sure it's not relevant for single-channel mode. I originally had it as 0.005, but then changed it during the period when the wrong frequency was being used. For cartographer it's whatever the firmware there sets. For mellow fly it should also be 0.005, but I was also confused by whether this even matters for single channel mode. If that 0.005 was actually a wait period, then you couldn't get more than 200 samples per second. Alternatively, that's the max time to wait for frequencies to settle, and typically everything settles much sooner... either way, in single channel mode, I think it's just continuously sampling.

The rcount/samples per second is the more interesting one. The datasheet says that the lower the sample rate the better the resolution. I've left the default at 250, but I run with 500 -- my logic is that at 3mm/s movement (my default tap speed), at a 250 data rate, the toolhead moves 0.012mm between samples. I wanted something lower. However as I type this out, this would be an interesting thing to ask people who have severe noise issues to change... I have very little noise from the sensor in my printer, but I've seen others where it's much worse.

[gloomyandy]

Hi, thanks for the reply!

I've been running my Fly boards with a divisor of 2 (so fRef of 20MHz), after seeing the notes in the datasheet I hadn't considered just running it at 40MHz. What is interesting is that the test conditions for many of the tables and graphs in the datasheet specify running with a 40Mhz Xtal, a divisor of 1 and using continuous mode which seem a bit of a contradiction. Looking at this app note: https://www.ti.com/lit/an/snoa944/snoa944.pdf?ts=1738918146028 that also refers to the 35MHz limit and even provides a way to work around it by using a "dummy" channel. I'm not sure if that solution is possible though if no coil is attached to the other coil interface (as I assume it would generate errors, that may cause other issues).

I think the reference to 55MHz may come from the possibility of the internal clock reference which seems to have a pretty wide clock range (but it may not be very stable). For now I will probably stick with 20MHz but once I have that stable I may well experiment a little.

I'm also planning on experimenting with different sample rates. My Fly setup seems to suffer with a fair bit of jitter. I'm not totally sure what the source of that is, but it may be from the fairly long flex cable used to attach the coil. Another possibility is that the prototype coil board I'm using is contributing to the problem.

The current RRF code (which is here: https://github.com/Duet3D/Duet3Expansion/blob/3.6-dev/src/CommandProcessing/ScanningSensorHandler.cpp) makes use of a simple moving average of "velocity" values. However I've been seeing a lot of false positives with that code and have been working on alternatives. I'm keen to try the butterworth filter that you are using to see how well that works.

Thanks for your work on this I will update things if/when I test more.

[gloomyandy]

Not directly related to this issue, but I have a version of RRF now running that uses a Butterworth filter. This seems to be working well in terms of detecting the touch and ignoring false positives, but I've noticed that compared to the other methods I've tried the detection point is delayed (resulting in more pressure on the bed), lowering the touch speed helps considerably with this for me. At the moment I'm running at 100mm/min (1.6mm/s) and that seems to work really well, I started off running at 300mm/min (5mm/s), but at that speed I was seeing considerably overshoot (up to 0.15mm), I wonder if you saw the same?

I'm also very curious as to how/why you came to use the Butterworth filter for this and would love to better understand how it actually works in this application!

[vvuk]

Hmm interesting! What other methods have you tried? I originally went with a somewhat complicated approach:

• compute a windowed moving average of the frequency
• compute the derivative of the wma
• compute a wma of the derivative (with a smaller window)
• look at the derivative of that, and trigger if a constant positive derivative exceeds a threshold; reset if it ever goes negative

This worked, but had some problems. I came up with this by starting at a bunch of graphs (I assume you've seen the graphs that taps produce?) and trying different things largely by intuition. It's still available as mode wma.

One of the users on the Sovol discord (Sesse) has some signal analysis background, and he suggested a different filter might give a clearer signal because ultimately what we're looking for is an inflection point, where the (mostly) smooth frequency suddenly stops changing nearly as rapidly. We experimented with a few things, but a 2nd order butterworth bandpass filter seemed to give pretty reliable results, including on machines that are super noisy. I have a stock Ender 3 I was testing on that has the rigidity of a wet noodle, and it worked well there, as well as on my sv08. It's possible to tweak those filter parameters (filter order, bandpass frequencies etc) which could give better results.

What detection methods have you tried that give better results? I'm happy to implement them as well for comparison.

I admit I haven't actually tried slower speeds. I stuck with 3mm/s as what I thought was a fairly conservative speed.. and intuitively (but perhaps incorrectly) I figured a faster speed would make the inflection point much more obvious (at the risk of needing to target a lower Z value, because the toolhead can't decelerate).

As for overshoot -- do you mean missing the contact point by that much, or the toolhead continuing to move that much further beyond the contact point before the tap is detected? The code has this tap_adjust_z parameter which is a constant value to add to the detected tap point; on my printers I need about +0.025 (sv08) to +0.150 (wet noodle) in order to get Z where I want it after a tap, but it's a constant factor regardless of temps etc. If the toolhead continuing to move before the tap is triggered (even though the tap start point is before that).

There's still a few things I don't actually understand; for example, on many machines, there's a "bump" peak in frequencies as the toolhead approaches the contact point. You can see it a bit in this graph here:

where right before the tap start the Z line (which is computed directly from the frequency) takes a sharper dip. I don't know enough about the physics involved to know if this is expected as the distance between sensor and target surface gets really small (and if e.g. moving the sensor a bit higher would help).

[gloomyandy]

Hi, thanks very much for the information that is very useful, the graph especially so!

In terms of overshoot, I think the Butterworth filter method must be triggering slightly later than the other filters I've been using as this is really the only variable that changes. So for instance in my test setup the older filters (see below) when running at 300mm/min trigger at a height of -0.54mm but the Butterworth will trigger at around -0.69mm. At slower speeds they are much closer. I suspect this may be partly due to how we test for the trigger, in the Butterworth case we are waiting for an upturn in the filter output. I'm not sure this is a problem though as it does look like this filter is very good at reject noise and does provide very solid triggering.

I've tested two other filters, the first used a simple moving average of the rate of change of the probe output and compared this reading against an average of the last four readings. This would work but was very susceptible to noise. The second filter was similar to your wma design, in that it compared the current and previous outputs of the moving average, but in this case the moving average was fed by an average of the four latest probe "velocities", this seemed to handle noise better, but I'd say it was not as good as the Butterworth filter in rejecting noise. Neither of these two filters seem to work as well as the Butterworth at lower speeds.The code for all of this is here: https://github.com/gloomyandy/Duet3Expansion/blob/v3.6-touch/src/CommandProcessing/ScanningSensorHandler.cpp#L148

On my test setup (an e3d toolchanger) a problem I see is there these seems to be some small "rough spots" in the movement of the bed that generate what looks like small pauses in the output from the probe. I'm not sure if this is caused by the threaded rod drive or some sort of "sticktion" they only last a very short amount of time (around 8 microsteps), but they can cause false positives. I'm pretty sure this is not electrical or sensor noise as the the position in Z that these false positives occur is pretty repeatable.

[gloomyandy]

Hi a quick follow up to the above. I've been investigating the overshoot I mentioned previously. One thing I don't really understand is why the trigger detection code needs to wait for the output of the filter to change from falling to rising before it tests against the threshold. This just seems to delay the trigger detection point (which seems to be a bad thing). I've some experimental code that moves the detection logic to be active while the filter output is falling and with this the touch detection logic seems to trigger earlier (as you would expect) and it does not seem to suffer from any more false triggers than the existing code. You can see my experimental code here: https://github.com/gloomyandy/Duet3Expansion/blob/v3.6-touch/src/CommandProcessing/ScanningSensorHandler.cpp#L172

I'd be interested to understand why the current detection code waits for the filter output to start to rise, are there situation in which doing that is important?

[vvuk]

One thing I don't really understand is why the trigger detection code needs to wait for the output of the filter to change from falling to rising before it tests against the threshold

Oh wow, this is a bug! It's supposed to continuously test while the output is falling and triggers when the threshold is hit. I just took a look at the code -- I did some refactoring in here and screwed it up. Thank you!

I wish it wasn't difficult to write standalone tests for all this.

[vvuk]

Some additional observations (this turned into a discussion thread, but whatever :)).. I just took a close look at varying speed. My observation is that lower speed is very noisy and it's difficult to detect the accurate tap point. At higher speeds, tap in theory is detected more accurately. This intuitively makes sense to me, because the inflection angle is more acute the faster the toolhead is moving. But I think because the tap detection only happens at the sampling rate, the tap start time will always fall on one of those slots.. and at 5mm/s and 500 samples/sec, the toolhead moves 0.010 between samples which is quite a bit.

The graphs look much cleaner at 5mm/s, but with more overshoot and the inaccuracy in values I mentioned above:

At 3mm/s (the default):

And at 1mm/s, though still detected fairly well:

[gloomyandy]

Hi, yes sorry for taking this off topic, I'd be happy to move this to some other place if you would rather, but I'm finding the discussion very helpful!

Yes I agree that slower speeds are definitely noisier and harder to deal with. But they do have the advantage that the energy involved in the touch is lower!

Looking at the graphs I'm curious as to why your touch detection logic is not treating the large rise/fall at the left end of the graph as a touch event? It looks like the threshold for detection has been exceeded.

Also I see that you have modified the touch detection logic. One question on that. Is there a reason you use "last_value" to obtain "diff" rather than using "val"? It seems to me that using val would trigger a little sooner and would also avoid the need for swapping the sign of diff (as you know at that point the val must be less than last_val, which I think means it will always be lower than start_val)?

Thanks again for taking the time to explain/look at this.

[vvuk]

(No worries about off topic, I'm the only owner of this repo, so whatever)

Looking at the graphs I'm curious as to why your touch detection logic is not treating the large rise/fall at the left end of the graph as a touch event? It looks like the threshold for detection has been exceeded.

At the start of filtering, the filter sees a sharp discontinuity between 0 (initialization values) and the first real value which causes that strong response. There's ways to avoid it by computing a steady-state initialization but it's complex. So, instead, I start filtering early (after the first "safe start" frequency, which is the same as home trigger height (2.0mm) but don't do any threshold checks until the second "safe start" frequency which is hardcoded to 1.5mm. That's actually cutting it close and I should probably drop it down to 1.3mm or so.

The "safe start" bits are there to make sure that the toolhead actually started high enough (above them) and that it moved through them while homing/tapping to make sure we get clean data from a clean movement and that we didn't try to home with the toolhead 1mm off the bed or something. (For homing, there's only one, and it's the home trigger height + 1.5mm if I remember.)

Also I see that you have modified the touch detection logic. One question on that. Is there a reason you use "last_value" to obtain "diff" rather than using "val"? It seems to me that using val would trigger a little sooner and would also avoid the need for swapping the sign of diff (as you know at that point the val must be less than last_val, which I think means it will always be lower than start_val)?

... that's because I was trying to fix the issue quickly and didn't think it through too clearly. In my defence, I'm getting over a flu that knocked me out for a few days ;) Good catch again, I should get you to review my PRs 😂

(The diff sign flip was me thinking about future-proofing; you can in theory specify any filter you can write in sos form, and for some of them you may want a raise rather than a fall. But of course the code only checks for the fall, so there's no point.)