Utilizing multiple piezo sensors for positional sensing

[corrados]

The Roland PDA120LS mesh head pad is equipped with three piezo sensors which are evenly distributed close to the edge of the pad. The original wiring of these piezos is to combine their signal into one trigger output signal, i.e., Roland does not evaluate each piezo signal separately.

Since Roland uses multiple piezo sensors in their PD-140DS digital snare pad to improve the positional sensing and also eliminate the hot spot behavior of center mounted piezo sensors, it makes sense to try to explore this possibility for Edrumulus as well.

To output the individual piezo sensor signals, I have disconnected two of the piezos from the Roland PCB and extracted the signals to a separate audio jack connector:

On the Git branch dual_meash_head_trigger_tests, I have done a simple test where I run the complete Edrumulus peak detection algorithm on both signals and did a comparison of the detected first peak positions:

In this example, I have varied the position of the hit on the mesh head by moving from one edge to the other and I have done it four times with rotating 90 degrees further for each iteration.
It is clearly visible that even without interpolation (which would improve the precision of the peak detection), the time difference in samples is a clear indicator of the position of the strike on the mesh head.

[corrados]

I have recorded new data, now striking the head directly from one sensor to the other and in a second run orthogonal to the first run:

Here, we can clearly see that the algorithm works very well. In the first half of the trace we have a dynamic of about 27 samples and in the second half (i.e., always in between the two sensors) we are always in the center as expected. And the resolution of one sample is definitely sufficient to get a good precision of the position.
The challenge to get a good working multiple sensor position detection is a very reliable first peak detection. Also, the mathematics to calculate the position on the mesh head if we only have the two time differences available is a complicated task to solve...

[jstma]

Naturally I'm interested in this because if you can accurately locate the position of the strike, you can do far more than simple positional sensing. You could, for example, split the drum head (or a suitably modified practice pad) into multipe zones for different sounds.

I don't fully understand your plots though.

I created a simple simulation in octave to compute the combined piezo signal based on simple time delays. It's simple because it assumes that the wavefront propagates from the strike point to the piezos without a change in amplitude or shape.

However, I don't know what the impact looks like nor what the propagation speed of the mesh is. So I eyeballed both:

If the speed of propagation is high, then the peaks are too close together to resolve separately. In fact, they overlap and combine constructively to produce a big signal. I had the speed of propagation really low to get separate peaks in the plots above. Only 10m/s. This is unrealistic. The chances are good that the speed of propagation will be higher than the speed of sound in air. I saw one reference that a mylar drum head speed of propagation is 1500m/s. At that speed there are very few samples of delay between the piezos even at 48kHz.

To improve the model you could take some piezo recordings. Use one piezo as a reference and hit directly on top of it. We can measure the propagation time from the recording and using the distance between the piezos we can calculate the propagation speed.

Can also fine tune the fake impact to look more like what the piezo sees. In fact, we could just subsitute a real recording.

Here is the octave code 3piezo-position.zip. I don't know if it's useful to you. You can modify it for two piezos if you want. Should make it easier to test any algorithms with.

However, the problem with edge mounted piezos is that the impact wave also travels to the edge of the rim and then reflects back. So at certain strike positions the reflection may reach the piezo before the main impact wave does. One way to deal with this might be to dampen the mesh and I know various edrum manufacturers do this differently.

I found this (video)[https://www.youtube.com/watch?v=2b89V7kAbZ0] on youtube on how stuffing a pillow in the drum does the same thing for an edge mounted trigger. It also makes the mesh drum head less bouncy. I tried this out breifly on one of my toms (no triggers) and it feels much better than a single ply mesh head.

[corrados]

Also note that my detected position and the raw wave signal correspond on the time axis like this:

[jstma]

If I understand what you're trying to say, you want to detect the time difference between the arrival of the strike at each of the piezos. You then want to use this time difference to estimate position. I also assume that for positional sensing, we're actually talking about concentric circles around the centre of the drum head. If that makes sense.

I made this illustration to show you that if you go by time of arrival, you can't actually tell where the strike is. I'm making the assumption that if the distances are equal, the time of arrival are equal. This may not be entirely true, but it's a good place to start.

I've drawn a drum head and two piezos (the small circles). The strike/impact is the green dot. The blue lines are the distance to each piezo and they are constrained to be equal to each other. The centre line shows the path along which the time of arrival will be equal (ie. the blue lines are equal length).

The case of equal lengths is easy to draw, but I can also constrain the lengths of each piezo distance to be a constant difference. This allows us to map a constant time difference. As you can hopefully see in the next two images, a constant difference of 0.5" and 2" between the lines follows the "arcs" shown.

What you end up with kinda looks like a basketball or a smith chart maybe. It doesn't look like the concentric circles of what I expect for positional sensing.

[corrados]

You are one step ahead. The initial post was about a proof of concept that it is possible to detect the position in between two sensors based on the time difference measurements of these two sensors. Detecting the position on a mesh head using three sensors is not trivial. It is a non-linear problem which can be solved with an iterative Newton method. We have created an initial Octave test file for the algorithm. Unfortunately, we still have convergence problems and the investigation is still ongoing.
On my Git test branch for positional sensing with three sensors, I have implemented a very rough approximation by using r = max(|L21|, |L31|, |L32|), where L21 is proportional to the measured time difference between sensor 2 and 1. This works surprisingly well.

[jstma]

Detecting the time difference between two sensors is nothing new. It's done all the time in radio and microphone arrays. How to do that efficiently is an issue, but the most robust way is cross correlation. With piezos we tend to have a very low noise floor so some form of threshold could work.

Do you have a recording for the 3 piezo test? I don't see it on that branch. I'd rather play around with a 3 channel recording than 2.

[corrados]

Unfortunately, I cannot record audio with 3 channels. I only have sound cards available which can record 2 channels only.

[corrados]

Here is a status update:

Solving the double trigger issue

To overcome the issue that we can get double triggers in case we do a peak detection on each sensor separately, I have now introduced a fourth channel which is the sum of all three sensors. I now perform a peak detection on that sum signal which eliminates the double trigger issue. The additional individual sensor streams, I only use for positional sensing and hot spot elimination. That seems to work ok. One draw back of that algorithm is that since I have to wait for the detected peaks of the individual sensors, I have to introduce some additional delay (in the order of 2 ms).

Determine the strike position on the mesh head based on the measured time differences of the detected peak

It turned out that the problem of finding the position based on measured time differences is a classic Hyperbolic Navigation problem. This is, e.g., utilized in the LORAN , Long Range Navigation: quote: "The time difference between receptions of pulses from several stations establishes a hyperbolic position line, which may be identified from a LORAN chart. A fix can be obtained by drawing two hyperbolic position lines."

[jstma]

What a coincidence. Just today I am playing with this hyperbolic trilateration. I found an interesting project written in something called "processing":

https://github.com/adamlamine/Piezo_Trilateration_Visual

It doesn't try to solve the problem, but rather plots the hyperbolas and tries to find the intersection by iterating over the search space. I was looking for a way to do just this in a graphical way and this processing (java) project is a good start.

Unfortunately it a few problems. It has 4 piezos placed and only uses 3 of them. It hardcodes the relationship between the hyperbola axes: one vertical and one horizontal. It's not written very nicely. And it's java ;p

There is a good run down on the formulas here including how to solve it.

https://math.stackexchange.com/questions/3373011/how-to-solve-this-system-of-hyperbola-equations

I haven't tried it yet, still trying to fix that guy's piezo project so that it can identify the intersection of the hyperbolas. I may give up and move on to a 3 piezo solution. The challenge here is to place one of the hyperbolas on an angle (so not vertical or horizontal).

[jstma]

Another thing I want to try comes from beamforming. The idea here is to find the angle to the source using the time delay. This would effectively plot two lines that should intersect and might be an easier solution. The issue with this approach is that it assumes plane waves which is a reasonable approximation to a spherical waveform at some distance away. With this particular 3 piezo configuration, I don't think this assumption of plane waves applies very well, but the error may be small enough to be usable.

[jstma]

The angle method doesn't work. I think it may be because the spacing between the piezos is large compared to the distance from the piezo to the impact.

The angle is a function of the time difference, you get one angle for one time difference. The problem is that you can produce said time difference between two sensors in multiple places.

So it's back to trilateration. Just to show you what the processing program looks like, here I'm plotting the mouse click (impact location) with a yellow dot. The green circles are piezos. The circle lines intersecting at the yellow dot are the radii based on time.

Just to be clear it's not a solution, just a statement of the problem.

[jstma]

Using that stackoverflow math, I was able to make trilateration work based on that processing framework I also linked to before. Here's a video of it working:

recording-2022-10-30_19.31.54.mp4

mouse click is the yellow circle, the red circle is where the algorithm thinks the yellow circle is.

There's a fair bit of math involved (lots of multiplies and 6 divides) and I'm not sure the esp32 can handle it. I'll post the code somewhere later today or tomorrow. For now the important part is:

	float r1;
	float r2;

	float x0;
	float y0;
	float x1;
	float y1;
	float x2;
	float y2;

	float dt0 = piezoList.get(0).deltaTime*soundSpeed;
	float dt1 = piezoList.get(1).deltaTime*soundSpeed;
	float dt2 = piezoList.get(2).deltaTime*soundSpeed;

	/* one of these dt is going to be r=0 and it will be the one closest to
	 * the point of impact.  We don't want to use that one.
	 */

	if(dt0 == 0) {
		r1 = dt1;
		r2 = dt2;

		x0 = piezoList.get(0).xPos;
		y0 = piezoList.get(0).yPos;
		x1 = piezoList.get(1).xPos;
		y1 = piezoList.get(1).yPos;
		x2 = piezoList.get(2).xPos;
		y2 = piezoList.get(2).yPos;
	} else if (dt1 == 0) {
		r1 = dt0;
		r2 = dt2;

		x0 = piezoList.get(1).xPos;
		y0 = piezoList.get(1).yPos;
		x1 = piezoList.get(0).xPos;
		y1 = piezoList.get(0).yPos;
		x2 = piezoList.get(2).xPos;
		y2 = piezoList.get(2).yPos;
	} else { /* dt2 == 0 */
		r1 = dt0;
		r2 = dt1;

		x0 = piezoList.get(2).xPos;
		y0 = piezoList.get(2).yPos;
		x1 = piezoList.get(0).xPos;
		y1 = piezoList.get(0).yPos;
		x2 = piezoList.get(1).xPos;
		y2 = piezoList.get(1).yPos;
	}

	float a1 = 2*(x0 - x1);
	float b1 = 2*(y0 - y1);
	float c1 = pow(r1, 2) + pow(x0, 2) + pow(y0, 2) - pow(x1, 2) - pow(y1, 2);

	float a2 = 2*(x0 - x2);
	float b2 = 2*(y0 - y2);
	float c2 = pow(r2, 2) + pow(x0, 2) + pow(y0, 2) - pow(x2, 2) - pow(y2, 2);

	float d1 = (2*r1*b2 - 2*r2*b1)/(a1*b2 - a2*b1);
	float e1 = (c1*b2 - c2*b1)/(a1*b2 - a2*b1);
	float d2 = (2*r1*a2 - 2*r2*a1)/(a2*b1 - a1*b2);
	float e2 = (c1*a2 - c2*a1)/(a2*b1 - a1*b2);

	float a = pow(d1, 2) + pow(d2, 2) - 1;
	float b = 2*(e1 - x0)*d1 + 2*(e2 - y0)*d2;
	float c = pow((e1 - x0), 2) + pow((e2 - y0), 2);

	/* two solutions to the quadratic equation */
	float r_1 = (-b + sqrt(pow(b, 2) - 4*a*c))/(2*a);
	float r_2 = (-b - sqrt(pow(b, 2) - 4*a*c))/(2*a);


	/* This solution is always wrong */
/*
	float x = d1*r_1 + e1;
	float y = d2*r_1 + e2;

	fill(0,0,255,125);
	circle(x, y, 10);
*/

	/* This solution seems to always be correct */
	float x = d1*r_2 + e1;
	float y = d2*r_2 + e2;

	fill(255,0,0,125);
	circle(x, y, 10);

[jstma]

I forked the original code and made changes to it here in case anyone wants to play with it.

[matthisfun]

corrados introduced me to the problem.

The asterisks are the unknown points. The diamonds are calculated by a C++ algorithm published here
https://github.com/matthisfun/det_pos

[jstma]

I have ported your C++ code for calculating the position to Python and added it to my script.

I assume this wasn't directed at me? I posted some java code.

I then mounted a projector above the snare pad to show the detected position right at the pad. Here are the results:
threesensor2.mp4

This was pretty cool! How did you get the data from the snare into python?

Please note that this is an initial test without having spent too much time into calibrating the system. You can see that the detected position is a bit off but I assume this can be fixed by some smart calibration. What does not work right now is the left half of the pad. If I play at the left half, the detected position is incorrect. I have to investigate why this is the case.

If you happen to be talking about my code, I saw something similar when I first implemented the trilateration math. It was important not to use the closest piezo (ie. the zero arrival time) in the rest of the calculation. It's because that closest piezo is the radius being solved for.

Anyway, this was a funny experiment :-)

That was pretty clever.

[jstma]

Actually, the third distance is redundant. If you have two of them, you can derive the third difference from the two others, e.g., l21 = l31 - l32 proof: l21 = l3 - l1 - (l3 - l2) = l2 - l1

I wanted to respond to this with some drawings.

In the figure below, the solid blue lines are equal distances from the impact point. This distance is unknown and it's the same distance from the impact point to piezo1 which is the first to trigger. The dashed lines continuing to the other piezos represents the time of arrival delay, expressed as length.

When we convert the time of arrival difference into a distance, it's fairly straightforward to understand what the differences of l21 and l31 mean, assuming piezo 1 is the first piezo to trigger. What I'm trying to understand is how, geometrically, the difference of l32 can be depicted. In the above photo, your algorithm picks out l21 as the biggest. The circle then becomes a radius 70% of l21. I've shown it below:

However, no matter what I do, I can't get an l32 that is bigger than l21 or l31 when piezo1 is the first piezo to trigger. For l32 to be big enough to do anything, the first piezo to trigger has to be either 2 or 3. This is the same as rotating the drawing. Which is what I believe your taking the maximum amounts to.

So l32 is never used when piezo1 is the first to trigger. And so there's no need to discuss what it might actually mean in a geometric sense.

And this is basically how the trilateration algorithm works. But instead of taking 70% of the largest distance, it calculates the circles that will intersect. You can see this below. The two solutions to the quadratic equation are outlined by the green circles. The dashed circles form the intersections we are looking for.

It may not look like it, but the unknown distance from the impact location to piezo1 is implicity part of the calculation because it is the zero time reference. It's important because we know where it is located.

As to why 70% works with your algorithm, that's a bit of a mystery. I mean you could take the average of l21 and l31 and get a similar result. It would just over/under estimate in a different way.

[corrados]

I have ported your C++ code for calculating the position to Python and added it to my script.

I assume this wasn't directed at me? I posted some java code.

Yes, it was. I was referring to your code in your previous post. Isn't that C++ code?

How did you get the data from the snare into python?

I am exporting the three values from Edrumulus using the Arduino Serial.println function. In Python, I simply read a line from the serial interface and do some value conversions. That's basically it.

What I'm trying to understand is how, geometrically, the difference of l32 can be depicted. In the above photo, your algorithm picks out l21 as the biggest. The circle then becomes a radius 70% of l21.

I must admit that I did not spend much time on deriving that simple approximation algorithm. It was really just a "quick hack" to be able to quickly test something. I was not expecting that algorithm to perform well at all, actually.

[jstma]

Yes, it was. I was referring to your code in your previous post. Isn't that C++ code?

Nope. It a language called Processing which is transpiled into of Java. Basically, from what I understand, it removes the need to specify Math.pow and you can use pow instead. It also has a built in canvas so you can just use the function circle rather than specifying a specific module. So I guess it simply "pollutes" the nsamespaces for the sake of syntactic sugar.

It has a setup and loop function, which in this case is the drawing loop, but it reminds me a lot of arduino. For this particular problem, I find it much nicer to use than python.

How did you get the data from the snare into python?

I am exporting the three values from Edrumulus using the Arduino Serial.println function. In Python, I simply read a line from the serial interface and do some value conversions. That's basically it.

Interesting. So if I put some piezos on my practice pad, I could do something similar with a breadboard, some jacks, some resistors and an esp32?

What I'm trying to understand is how, geometrically, the difference of l32 can be depicted. In the above photo, your algorithm picks out l21 as the biggest. The circle then becomes a radius 70% of l21.

I must admit that I did not spend much time on deriving that simple approximation algorithm. It was really just a "quick hack" to be able to quickly test something. I was not expecting that algorithm to perform well at all, actually.

I have a lingering doubt that your approximation might be an answer to a different question. And that if we ask that question properly, we can maybe derive a solution that's easier to compute than the full quadratic solution. I mean you don't need to know the precise location, you just need to know the radius of the circle that lands on that point. I don't know if that makes the problem easier to solve. Anyway, I'll let that purcolate in the background. If I make any major discoveries, you'll be the first to know ;)

[corrados]

So if I put some piezos on my practice pad, I could do something similar with a breadboard, some jacks, some resistors and an esp32?

In theory, yes. But the code is still very experimental. I am working on a Git branch sum_three_sensors_as_detection. I am using my Prototype 4 and use the Snare input plus the Ride (6) input for piezos 2 and 3.

[corrados]

I have created a short Youbube video today, showing the latest algorithm performance.

[jstma]

Is there a way to make a similar recording, but also display the number of samples between arrivals on screen somewhere? Some of those hits don't register close enough to to the source. I'm curious as to why.

[corrados]

I was checking the number of samples between arrivals during my calibration for all possible positions on the pad. For the calibration, I had to introduce some "magic offset" to one distance which is hard to explain since the maximum possible offset introduced by the ADC is one sample but the "magic offset" was several samples. Also, if I strike far away from a sensor, the error in time difference measurement is large. I also checked that your algorithm works perfectly. In that file pos_det.py, you can click on the graph with the mouse and the detected position using your algorithm is shown. This is always in perfect sync with the mouse click. Therefore, the problem is not your algorithm but the distance measurement.

I suspect that my peak detection algorithm lock onto an incorrect peak if the strike is far away from the sensor. Further investigations in this regard are needed to find the real cause of the problem.

[peteallenm]

I'm really interested in this.
Did anyone ever try with cymbals rather than mesh drums?
Is this what the Roland 'digital' cymbals use?

[corrados]

I haven't used it with cymbals. Just with my mesh pad. And I do not know how Roland does it.