Fixes for SYSEX channel messages / Added master volume method (also for SYSEX)

[dcoredump]

Now it is possible to SYSEX edit only one voice (no need to set the voice to MIDI channel OMNI anymore).

Added sending and receiving of MIDI SYSEX voice dumps. A dump is send when changing the voice, so an editor can show automaticly the latest voice configuration.

Added SYSEX global master volume and master volume handling inside MiniDexed.

[probonopd]

That's great, thank you very much @dcoredump.
Let's test: MiniDexed_2022-05-22-86d63bf

[probonopd]

Here are my test results so far:

• Sending voices from Dexed to MiniDexed works great
• Sending whole cartridges of voices from Dexed to MiniDexed does not work (likely not implemented in MiniDexed yet)
• Dexed doesn't seem to receive voices from MiniDexed (likely not implemented in Dexed yet)
• Could not test with Edisyn (Edisyn worked exactly once for me; since then it always says "There are no MIDI devices to receive from" (likely a Edisyn bug)
• How do I use Master Volume?

[miotislucifugis]

Nice work. Seems to be working for the most part but Ive experienced some delay with sending individual parameter changes while playing (or at least holding a note.) Seems good when I first start, but over time messages seem to get missed or there is a long delay/ backlog that builds up and the parameter is still changing after I stop moving the dexed knob.

[probonopd]

@miotislucifugis can you describe exactly each step you are doing so that we can reproduce this? Thanks.

[miotislucifugis]

Yeah- let me check again with a different (better) midi usb interface to rule that out...

[dcoredump]

Thanks for testing!

Indeed: for whole cartridges the code for recognation exists, but storing them is not implemented. Therefore a way for naming and storing is needed.

EdiSyn seems not to recognize a voice bulk dump and Dexed maybe also not. But Herr Müller sends and accepts them.

Why changing the parameters get stuck after a while is really strange. I will take a look at this.

[Edit] Master volume works like mentioned in
#157. I have never seen that this is implemented in any hard- or software.

[miotislucifugis]

Ok its doing it with my MOTU interface... using Dexed, here is what Im doing:
I think it is best heard by making a big change on one of the dexed knobs- im using Algorithm because it makes a easy to identify audible change. Turn the knob quickly from minimum to max position, algorithm1-32 while playing a note. I can here the sound step through each algorithm step by step, continuing for a moment or two after I stop turning the knob. I dont remember exactly what I did before , but I think that after changing algorithm, I immediately started adjusting LFO speed and since I didnt hear an immediate change (algorithm messages were still being processed) I wiggled the LFO speed knob even more, which added even more sysex parameter change command to the queue. Basically, it seems like it is possible to turn knobs and send sysex faster than it can be processed. It is not instant like when sending midi CC messages.

[probonopd]

Note to self: Update MIDI implementation chart once this is merged.

MIDI Master Volume F0 7F 7F 04 01 ll mm F7
ll (LSM) = ignored
mm (MSB) = master volume level

[sou7611]

Question, and this may have to be answered by someone with real hardware : does a real DX7 continue to play if you change algorithms? Or does it reset back to note off?

…

On Sun, May 22, 2022, 2:30 PM probonopd ***@***.***> wrote: Note to self: Update MIDI implementation chart once this is merged. MIDI Master Volume F0 7F 7F 04 01 ll mm F7 ll (LSM) = ignored mm (MSB) = master volume level — Reply to this email directly, view it on GitHub <#249 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABNBVNBTNEKJ4UK65DG4GTLVLJ4MFANCNFSM5WTKLCCQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[miotislucifugis]

Question, and this may have to be answered by someone with real hardware : does a real DX7 continue to play if you change algorithms? Or does it reset back to note off?
…

I can check this out, probably tomorrow.

[dcoredump]

Question, and this may have to be answered by someone with real hardware :
does a real DX7 continue to play if you change algorithms? Or does it
reset back to note off?

On Sun, May 22, 2022, 2:30 PM probonopd @.***> wrote:

Note to self: Update MIDI implementation chart once this is merged.

MIDI Master Volume F0 7F 7F 04 01 ll mm F7
ll (LSM) = ignored
mm (MSB) = master volume level

Good question... I have a real DX7(II) but first I have to find a place for him...

[dcoredump]

I took a quick look at the code: the dispatching of MIDI events can be optimzed a little bit. But I think this would not speed it up enough. I don't know if there is something like a task priority in circle. Maybe locking will help. I have to test a little bit with this...

[miotislucifugis]

I tested with sending parameter changes from dexed (and edisyn) to a real dx7 and the behavior is not that different. Sysex messages are a bit delayed and it does not respond like instant cc commands. The DX7 may be a bit faster than minimexed when sending a lot of value changes (turning dial from min to max = less lag on the DX7) but the big difference is that the DX7 terminates the note being held and will not allow a retrigger until all the sysex command in the queue have been processed. As is, i think that I prefer the current behavior of miniDexed compared to the original synth, just with the caveat that one must take slow and steady with the knob tweaking.

[sou7611]

That's why I asked my question above, I suspected the possibility that DX would reset its internal computer when receiving/processing sysex, because parm changes = math changes, so the only way to do that is note off, and require the user to restrike a note to hear differences. Lesser important when tweaking oscillator settings, but definitely important when changing algorithms. The question to ask is, do we want to faithfully emulate the hardware here? If so this behavior needs to be replicated as well.

…

On Mon, May 23, 2022, 11:26 AM miotislucifugis ***@***.***> wrote: I tested with sending parameter changes from dexed (and edisyn) to a real dx7 and the behavior is not that different. Sysex messages are a bit delayed and it does not respond like instant cc commands. The DX7 may be a bit faster than minimexed when sending a lot of value changes (turning dial from min to max = less lag on the DX7) but the big difference is that the DX7 terminates the note being held and will not allow a retrigger until all the sysex command in the queue have been processed. As is, i think that I prefer the current behavior of miniDexed compared to the original synth, just with the caveat that one must take slow and steady with the knob tweaking. — Reply to this email directly, view it on GitHub <#249 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABNBVNEUJU3SUX437CGK5BLVLOPUFANCNFSM5WTKLCCQ> . You are receiving this because you commented.Message ID: ***@***.***>

[probonopd]

The question to ask is, do we want to faithfully emulate the hardware
here? If so this behavior needs to be replicated as well.

If we can do things today that the original hardware could not do back in the day, then we should take advantage of the fact that we are now almost four decades later. For example, MiniDexed allows you to change parameters using the rotary encoder and display while holding down a key pressed and hear immediate changes in places the DX7 required you to strike a key again. I'd assume that everyone would consider this as an advantage.

So, if we can do things better than the DX7, then we should. But at least we should not do any worse.

[sou7611]

True enough, although it has to be pointed out that could lead to math glitches, stuck notes, etc ... which is probably why Yamaha in some cases did indeed do a note-off.

…

On Mon, May 23, 2022, 1:23 PM probonopd ***@***.***> wrote: The question to ask is, do we want to faithfully emulate the hardware here? If so this behavior needs to be replicated as well. If we can do things today that the original hardware could not do back in the day, then we should take advantage of the fact that we are now almost four decades later. For example, MiniDexed allows you to change parameters using the rotary encoder and display while holding down a key pressed and hear immediate changes in places the DX7 required you to strike a key again. I'd assume that everyone would consider this as an advantage. — Reply to this email directly, view it on GitHub <#249 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABNBVNFIWZWXXAXBOMZRNVTVLO5JRANCNFSM5WTKLCCQ> . You are receiving this because you commented.Message ID: ***@***.***>

[dcoredump]

Thanks for testing @miotislucifugis!

As I see it there are two problems:

1. some parameter changes should disable all currently played notes. The implementation for this is simple. Just had to identify all parameters for it.
2. Too many parameter changes are flooding the engines. Here it becomes difficult. A parameter change is actually nothing computationally intensive. Only one value in an array is changed. So the problem could be a low priority in processing the data. At least all data seems to be buffered. The best solution would be, if the sender (e.g. Dexed) could already make a separation of the data quantity with fast controller movements.

[dcoredump]

I added a spin lock around the MIDI processing. Perhaps this may help? Can anyone test please?

[probonopd]

Thanks @dcoredump.
@miotislucifugis would be great if you could give this a test run:
MiniDexed_2022-05-24-fea0556

[miotislucifugis]

I had some time to do a quick test this morning, using MiniDexed_2022-05-24-add693c as that seemed to be the newest. Much improved! I noticed that the note is terminated when changing algorithm, but the change was very quick, even when rocking the knob back and forth a few times. With other parameters that I tested , LFO speed and some op eq times, the note was not killed and one could hear the changes while playing. it wasnt super CC# instantaneous, but it was reasonably fast and I did not encounter any of the sysex 'log jams' like before. i think that this is pretty good and very usable for general tweaking of patch parameters. I can see a potential scenario where there may be problems if someone tries to continuously modulate a parameter using a device where one can assign an lfo or something to a sysex command...
but there would probably also be similar problems if one tried to do that with a real DX7, so I would say that it is good, at least for now

[dcoredump]

Cool, many thanks @miotislucifugis for testing and reporting!!!

[probonopd]

Thank you very much @dcoredump and @miotislucifugis.

[diyelectromusic]

I added a spin lock around the MIDI processing. Perhaps this may help? Can anyone test please?

Do we know the reason for the spinlock? I've noticed that all LOG messages will fail once acquired and was having a conversation about spinlocks in circle (more here), but having seen Rene's answer and descriptions of how they work, I'm now wondering if that perhaps isn't quite the right thing to be using here?

Thanks,
Kevin

[probonopd]

From the comments in this thread, I think it was added to make the processing of MIDI CCs faster/less laggy?
@miotislucifugis noticed improvements. Hopefully those weren't just due to the (now) lack of debug messages...

[diyelectromusic]

Well I guess that was partly my point... I was working through the discussion, but I didn't spot anything that indicated why adding some kind of thread syncrhonisation might solve the issue. If anything you'd think it would slow things down. In some cases that can improve performance (e.g. in networking if slowing things down prevents message retries or something like that) but I'm not sure why that would improve performance in MIDI.

If the problem was corrupt messages then that might have been due to interleaving of MIDI message bytes from different sources (serial, USB and keyboard for example), but again I don't think that was being discussed.

There is a very real possibility that the spinlock has had a beneficial side effect rather than being required itself, but I just don't know at the moment, hence asking :)

Whatever the reason, any LOG calls after the acquire are being ignored (and there are a few). I only noticed as I put some extra ones in for my own debugging and then spent the better part of an afternoon trying to work out why whatever I did kept resulting in no output!

Kevin