Support for Yamaha DX7 SysEx commands #195

dcoredump · 2022-05-01T12:59:06Z

Use this in combination with the latest Synth_Dexed (added a method called checkSystemExclusive()) which checks parameters, checksums, ... and returns a value for further actions).

This works for MicroDexed but I have no hardware for checking with MiniDexed.

SysEx master volume is currently not supported.

probonopd · 2022-05-01T13:10:20Z

Thanks @dcoredump. Great contribution!

Build available for testing:

https://github.com/probonopd/MiniDexed/suites/6332730588/artifacts/227887034

Testing it now.

probonopd · 2022-05-01T13:25:23Z

Dexed can send sysex of voices and whole cartridges:

Under Audio/MIDI Settings, the MIDI Output needs to be selected:

Edit: Also have to set DX 7 Out under PARM:

Now the sysex seemingly do get sent out to MiniDexed (the LED on the Raspberry Pi Pico MIDI device blinks) but MiniDexed seems to be rather unimpressed by it.

What should happen if a voice or a while cart is received by MiniDexed?

dcoredump · 2022-05-01T13:32:46Z

I am currently adding a DIN-MIDI to my MiniDexed, so I hope I can test SysEx also...

probonopd · 2022-05-01T13:37:37Z

This works for MicroDexed

How did you test it there, what software did you use on the PC?

dcoredump · 2022-05-01T16:02:08Z

This works for MicroDexed

How did you test it there, what software did you use on the PC?

I am using a Java software called EdiSyn.

But I do not get my DIN-MIDI to work as expected. I used a separate I/O module for 3.3V, but it seems not to work 😟.

Regards, Holger

dcoredump · 2022-05-01T16:10:40Z

Dexed can send sysex of voices and whole cartridges:

Yes, should work both, but for receiving a complete bank the code for storing is missing (look for TODO in mididevices.cpp).

dcoredump · 2022-05-01T19:02:24Z

I have found a problem that prevents the recognition of SYSEX messages. Maybe it works now. Too bad I can not test it.

probonopd · 2022-05-01T19:54:15Z

Unfortunately it seems that it doesn't. Do you have any chance to get access to a Raspberry Pi Pico?

dcoredump · 2022-05-02T06:47:42Z

Unfortunately it seems that it doesn't. Do you have any chance to get access to a Raspberry Pi Pico?

I think I still have one lying around somewhere (I keep little order in my electronics components). I'll have to look for it...

Or I have to fix my DIN-MIDI interface...

dcoredump · 2022-05-02T11:46:18Z

Unfortunately it seems that it doesn't.

Yes, and I know why. But unfortunately I won't have time to fix the error before tomorrow evening. It is of course a PEBCAK 😁

dcoredump · 2022-05-03T14:05:42Z

Ok, I hope THIS fixes the SYSEX problems. Currently no SYSEX generating hardware for testing here :(
I also hopefully added the SYSEX master volume but the main code for really doing the master volume change is not implemented.

probonopd · 2022-05-03T16:51:45Z

Sending a voice or a patch doesn't seem to do anything... but I feel we are real close now ;-)

dcoredump · 2022-05-03T18:00:08Z

Sending a voice or a patch doesn't seem to do anything... but I feel we are real close now ;-)

I just need test equipment. Hopefully I can manage this at weekend.

dcoredump · 2022-05-04T15:47:57Z

Sending a voice or a patch doesn't seem to do anything... but I feel we are real close now ;-)

... and I know why: CSerialMIDIDevice::Process() does not support SYSEX messages. They are filtered and it seems not very easy to add SYSEX handling... to fix this I first have to understand what this method does.

arsamus · 2022-05-04T19:23:58Z

Sending a voice or a patch doesn't seem to do anything... but I feel we are real close now ;-)

... and I know why: CSerialMIDIDevice::Process() does not support SYSEX messages. They are filtered and it seems not very easy to add SYSEX handling... to fix this I first have to understand what this method does.

As I've analyzed it last days, the method just accept standar 3 byte MIDI messages (Note On, Off, etc.), but not support for SysEx. It requiere a different algorhitm to handle SysEx and verify if the 100 bytes buffer is sufficient.

dcoredump · 2022-05-05T11:25:16Z

Sending a voice or a patch doesn't seem to do anything... but I feel we are real close now ;-)

... and I know why: CSerialMIDIDevice::Process() does not support SYSEX messages. They are filtered and it seems not very easy to add SYSEX handling... to fix this I first have to understand what this method does.

As I've analyzed it last days, the method just accept standar 3 byte MIDI messages (Note On, Off, etc.), but not support for SysEx. It requiere a different algorhitm to handle SysEx and verify if the 100 bytes buffer is sufficient.

Yep, just understand this now. I think I will extend the current algorithm simply by checking for 0xF0 setting a flag and storing all incoming data in an array when the flag is set, until 0xF7 occurs (or the highest bit of the incoming byte is set, what's the same) and than calling the SYSEX handling method.

dcoredump · 2022-05-07T07:14:12Z

I have now been able to narrow down the problem a bit more: Somewhere in the provisioning of data for CSerialMIDIDevice::Process() there seems to be problems and I think I cannot solve it without the help of @rsta.

In my latest commit I added some code for showing the incoming data right at the beginning of CSerialMIDIDevice::Process(). When I change the algorithm with EdiSyn I get the following:

Incoming MIDI data:
0000: 0xf0 0x43 0x10 0x01
Incoming MIDI data;
0000: 0x00 0x01 0x92

So far, it's ok - but there is a 0xf7 for the end of a SYSEX message missing.

When I am doing the same for MicroDexed the data dump shows:
[0xF0|240][0x43|067][0x10|016][0x01|001][0x06|006][0x0B|011][0xF7|247]

So I am pretty sure the last 0xf7 is sent by EdiSyn... I have no idea where to search for the msising byte.

TIA, Holger

probonopd · 2022-05-07T07:16:21Z

src/serialmididevice.cpp

@@ -59,6 +60,18 @@ void CSerialMIDIDevice::Process (void)
 		return;
 	}

+	printf("Incoming MIDI data:\n");


Let's do this only when MIDIDumpEnabled=1 in minidexed.ini, since on lower-powered RPi models logging can cause audio issues.

Let's do this only when MIDIDumpEnabled=1 in minidexed.ini, since on lower-powered RPi models logging can cause audio issues.

It's just fast debug code to find the problem... will be deleted later.

probonopd · 2022-05-07T07:17:55Z

In my tests 53b38dc causes MiniDexed to go entirely silent when voices or cartridges are sent to it via sysex from Dexed.

dcoredump · 2022-05-07T07:25:42Z

In my tests 53b38dc causes MiniDexed to go entirely silent when voices or cartridges are sent to it via sysex from Dexed.

Yes - it seems like the serial stack is filtering the 0xf7 byte (SYSEX end message).

probonopd · 2022-05-07T08:04:40Z

maybe @rsta2 knows?

rsta2 · 2022-05-07T09:31:46Z

The serial MIDI driver in MiniDexed filters out all MIDI system messages:

https://github.com/probonopd/MiniDexed/blob/main/src/serialmididevice.cpp#L73

The Circle serial device itself should be transparent and deliver all received bytes.

probonopd · 2022-05-07T15:08:56Z

serialmididevice.cpp:72:6: error: 'i' was not declared in this scope. Leftover?

dcoredump · 2022-05-07T16:03:19Z

serialmididevice.cpp:72:6: error: 'i' was not declared in this scope. Leftover?

Hope I fixed it. Cannot test currently... will do this later.

probonopd · 2022-05-07T16:08:32Z

Build succeeds but MiniDexed seems to do nothing with received sysex.

dcoredump · 2022-05-07T17:06:44Z

Build succeeds but MiniDexed seems to do nothing with received sysex.

My latest code first prints all incoming data (before any processing starts). There is currently never any 0xf7 shown, but EdiSyn is sending them (let's say: the same EdiSyn instance sends it to MicroDexed).

I currently do not not know what to do next. Is this a problem inside circle @rsta2? But why only 0xf7? Really strange...

arsamus · 2022-05-07T20:55:01Z

Hi @dcoredump how long is de Sysex message that you are sending? More than 100 bytes?

probonopd · 2022-05-07T21:02:08Z

Yes.

Voices have more than 100 bytes.

In the case of entire cartridges, significantly so.

dcoredump · 2022-05-08T04:59:23Z

Hi @dcoredump how long is de Sysex message that you are sending? More than 100 bytes?

No. Parameter changes are only a few bytes. And that's what I'm experimenting with. Also there are bigger messages (more than 128 bytes for a voice and 4104 bytes for a bank).

Furthermore the method should combine single message blocks to a whole message (as far as I understand), even if only small parts of a whole message are arriving.

probonopd · 2022-05-08T06:51:26Z

Parameter changes are only a few bytes. And that's what I'm experimenting with.

How are you doing that? I have not yet found a mode in which turning knobs in Dexed on the PC would always immediately send the respective parameter change CCs to MiniDexed, effectively turning it into a GUI for Dexed. Is there one?

dcoredump · 2022-05-08T07:53:21Z

Parameter changes are only a few bytes. And that's what I'm experimenting with.

How are you doing that? I have not yet found a mode in which turning knobs in Dexed on the PC would always immediately send the respective parameter change CCs to MiniDexed, effectively turning it into a GUI for Dexed. Is there one?

As I wrote some messages before: I am using EdiSyn and a cheap USB to serial adapter. This setup works for MicroDexed.

And I just figured out where the problem might be: If you look at the method CSerialMIDIDevice::Process() in serialmididevice.cpp, the method exits immediately if the return value is <= 0.

I have had printed the output and rarely get a -1 and if several bytes are sent a -3 back. According to include/circle/serial.h -1 means SERIAL_ERROR_BREAK and -3 SERIAL_ERROR_FRAMING.

I have a guess (@rsta): For some reason the serial driver does not handle large amounts of data at a time. It seems to be sufficient for the normal 3 bytes of a MIDI message, but for larger messages (e.g. a SYSEX parameter change of 7 bytes) it seems to have problems. So often the 0xf7 at the end of the message is gone - but often even less arrives.

I have no idea at all. I need someone who is familiar with circle's parameters. :(

My MIDI input circuit should be ok since @probonopd also seems to get only a few bytes of the message. @probonopd: If you ca manage to get EdiSyn running, perhaps you can also try my latest commit and take a look for the received data of a parameter change?

dcoredump · 2022-05-08T07:59:13Z

Hi @dcoredump how long is de Sysex message that you are sending? More than 100 bytes?

No. Parameter changes are only a few bytes. And that's what I'm experimenting with. Also there are bigger messages (more than 128 bytes for a voice and 4104 bytes for a bank).

Furthermore the method should combine single message blocks to a whole message (as far as I understand), even if only small parts of a whole message are arriving.

BTW: My information about SYSEX messages is from: sysex-format.txt

rsta2 · 2022-05-08T10:52:57Z

Very strange. These errors (-1 and -3) normally occur, when the serial connection hardware settings (31250 Bps, 8-N-1) aren't OK on the transmitting side. The serial driver has a 2048 Byte ring buffer for the received data, which is filled by a FIQ handler and continuously polled by MiniDexed. If it will overrun, an error -2 is returned.

dcoredump · 2022-05-08T11:36:29Z

Very strange. These errors (-1 and -3) normally occur, when the serial connection hardware settings (31250 Bps, 8-N-1) aren't OK on the transmitting side. The serial driver has a 2048 Byte ring buffer for the received data, which is filled by a FIQ handler and continuously polled by MiniDexed. If it will overrun, an error -2 is returned.

Thanks for the fast answer!

I am currently trying to fetch another USB-MIDI-Adapter for testing. But the one I used for MiniDexed works without problems for MicroDexed... perhaps a problem with my MIDI input circuit on MiniDexed?

My wife will kill me, but I need to buy a scope 😀

rsta2 · 2022-05-08T14:02:05Z

Are you sure, your MIDI input circuit provides 3.3V levels to GPIO15 (RXD), not 5V?

dcoredump · 2022-05-08T16:23:33Z

Are you sure, your MIDI input circuit provides 3.3V levels to GPIO15 (RXD), not 5V?

Yes, bought this. But I am not totaly sure if it's working as it should. Only a scope can help (I need to say this mantra as often as I can, so I may buy it 😀).

rsta2 · 2022-05-08T18:18:19Z

I just tried Edisyn, connected to a RPi 4B using this RPi-Pico-based MIDI-Adapter, with a simple Circle-based program, which only dumps the incoming MIDI data bytes. That is working well. I can see the 0xF0 and 0xF7 bytes, which start and end a Sysex transfer, when modifying some parameter in the DX7 patch editor. Edisyn sometimes sends a bunch of All sound off and All notes off messages, but the Sysex messages clearly arrive. Because this uses the same Circle serial driver, it should be OK.

dcoredump · 2022-05-09T05:17:11Z

I just tried Edisyn, connected to a RPi 4B using this RPi-Pico-based MIDI-Adapter, with a simple Circle-based program, which only dumps the incoming MIDI data bytes. That is working well. I can see the 0xF0 and 0xF7 bytes, which start and end a Sysex transfer, when modifying some parameter in the DX7 patch editor. Edisyn sometimes sends a bunch of All sound off and All notes off messages, but the Sysex messages clearly arrive. Because this uses the same Circle serial driver, it should be OK.

Many thanks for testing!!!

So I think my DIN-MIDI circuit has a problem. That's bad, but better than a bigger problem inside circle or MiniDexed.

Now I have to check what's going wrong with my MiniDexed and perhaps I have to solder a new DIN-MIDI circuit. Until than, I cannot test the SYSEX code 😟

probonopd · 2022-05-09T07:00:13Z

It works! It works! Using Edsyn, I can change e.g., the Transpose value.

But I get stuck notes like mad if I do it while playing.

craigyjp · 2022-05-09T07:06:10Z

Do you get stuck notes whilst editing from the front panel encoder too?

dcoredump · 2022-05-09T11:24:55Z

It works! It works! Using Edsyn, I can change e.g., the Transpose value.

But I get stuck notes like mad if I do it while playing.

Maybe a small problem inside Synth_Dexed. I will take a look a this soon.

[Edit] ~~You can try to comment out line 1112 in minidexed.cpp (doRefreshVoice()). Perhaps this line is not needed...~~ Have added this fix for testing to 1b681a8

probonopd · 2022-05-09T16:31:29Z

Do you get stuck notes whilst editing from the front panel encoder too?

Now that you are saying it: Yes. Will test 1b681a8 next.

probonopd · 2022-05-09T16:42:31Z

Still the case. To reproduce:

Using Chrome, go to https://synthmata.com/volca-fm/
From "Select MIDI Device", choose "TinyUSB Device MIDI 1"
Scroll all the way down
Play a note and while holding down the note, turn the "Transpose" knob
Release the key for the note
The note keeps playing forever until you switch the voice

Of course this is just a minor nitpick. Overall it is great that we can control MiniDexed via sysex messages. 👍

probonopd · 2022-05-09T17:04:04Z

Merging since this is a big improvement. Improvements can always be made in separate PRs.

This implements half of #82, namely "parameter changes".
What would need to be done to also address "voice loading via sysex" (voices and banks of voices)?

dcoredump · 2022-05-09T18:13:10Z

Merging since this is a big improvement. Improvements can always be made in separate PRs.

This implements half of #82, namely "parameter changes".
What would need to be done to also address "voice loading via sysex" (voices and banks of voices)?

The code exists. I cannot test, but I think it should work when you send a voice dump. For a bank dump code for storing the bank is needed.

I ordered a scope, hopefully to fix my problems with my DIN-MIDI circuit (ok, a Pico was cheaper, but if you please don't tell this my wife... thanks). After fixing I do not need write code without checking what I am doing anymore 😁

fritzvd · 2022-06-04T10:52:55Z

This is very cool. If it sends Sysex, this sounds like it can also work with: https://synthmata.com
Kind of an easy frontend to access all parameters 😎

Holger Wirtz and others added 2 commits May 1, 2022 14:51

Added SysEx handling.

db341ca

Merge branch 'probonopd:main' into main

ffa0bfd

probonopd changed the title ~~Supprot for Yamaha DX7 SysEx commands~~ Support for Yamaha DX7 SysEx commands May 1, 2022

Holger Wirtz added 2 commits May 1, 2022 20:53

Fix for SYSEX treatment.

c7ca66a

Fix.

b4a9c52

Fixes for SYSEX handling.

f68883b

Holger Wirtz and others added 2 commits May 5, 2022 17:23

Adding SYSEX handling to serialmididevice.

53b38dc

Added debuging code for showing all incoming MIDI messages.

fcc1eaa

probonopd reviewed May 7, 2022

View reviewed changes

probonopd and others added 2 commits May 7, 2022 15:09

Merge branch 'main' into main

7518c34

New fixes for SYSEX handling.

c41f02b

Fix for stopping stuck notes when editing parameters via SYSEX.

1b681a8

probonopd merged commit 3894cb3 into probonopd:main May 9, 2022

probonopd mentioned this pull request May 12, 2022

Serial Midi In double triggering notes #214

Closed

probonopd mentioned this pull request May 29, 2022

Cosmetic: 'mididevice: Send SYSEX voice dump to <garbled>' #262

Closed

Support for Yamaha DX7 SysEx commands #195

Support for Yamaha DX7 SysEx commands #195

Conversation

dcoredump commented May 1, 2022

probonopd commented May 1, 2022 • edited Loading

probonopd commented May 1, 2022 • edited Loading

dcoredump commented May 1, 2022 • edited Loading

probonopd commented May 1, 2022

dcoredump commented May 1, 2022

dcoredump commented May 1, 2022

dcoredump commented May 1, 2022

probonopd commented May 1, 2022

dcoredump commented May 2, 2022

dcoredump commented May 2, 2022

dcoredump commented May 3, 2022 • edited Loading

probonopd commented May 3, 2022

dcoredump commented May 3, 2022

dcoredump commented May 4, 2022

arsamus commented May 4, 2022

dcoredump commented May 5, 2022

dcoredump commented May 7, 2022 • edited Loading

probonopd May 7, 2022

Choose a reason for hiding this comment

dcoredump May 7, 2022

Choose a reason for hiding this comment

probonopd commented May 7, 2022

dcoredump commented May 7, 2022

probonopd commented May 7, 2022

rsta2 commented May 7, 2022

probonopd commented May 7, 2022

dcoredump commented May 7, 2022

probonopd commented May 7, 2022

dcoredump commented May 7, 2022 • edited Loading

arsamus commented May 7, 2022

probonopd commented May 7, 2022 • edited Loading

dcoredump commented May 8, 2022 • edited Loading

probonopd commented May 8, 2022 • edited Loading

dcoredump commented May 8, 2022 • edited Loading

dcoredump commented May 8, 2022

rsta2 commented May 8, 2022

dcoredump commented May 8, 2022

rsta2 commented May 8, 2022

dcoredump commented May 8, 2022

rsta2 commented May 8, 2022

dcoredump commented May 9, 2022

probonopd commented May 9, 2022

craigyjp commented May 9, 2022

dcoredump commented May 9, 2022 • edited Loading

probonopd commented May 9, 2022 • edited Loading

probonopd commented May 9, 2022 • edited Loading

probonopd commented May 9, 2022

dcoredump commented May 9, 2022

fritzvd commented Jun 4, 2022

probonopd commented May 1, 2022 •

edited

Loading

probonopd commented May 1, 2022 •

edited

Loading

dcoredump commented May 1, 2022 •

edited

Loading

dcoredump commented May 3, 2022 •

edited

Loading

dcoredump commented May 7, 2022 •

edited

Loading

dcoredump commented May 7, 2022 •

edited

Loading

probonopd commented May 7, 2022 •

edited

Loading

dcoredump commented May 8, 2022 •

edited

Loading

probonopd commented May 8, 2022 •

edited

Loading

dcoredump commented May 8, 2022 •

edited

Loading

dcoredump commented May 9, 2022 •

edited

Loading

probonopd commented May 9, 2022 •

edited

Loading

probonopd commented May 9, 2022 •

edited

Loading