Adding DMX512method to ESP32 I2S & I2Sx

[devarishi7]

As i asked you already in the old conversation But i understand a new conversation makes sense. It would probably need to structured a tad differently for the I2Sx method, the issue being the different way in which the output buffers are filled, and that within the same structure as all the other I2Sx methods hmm anyway i am also a little confused about the I2S sample rate for timing. The WS2812b method (as all others) uses 100000, but that is at 4 bytes per byte, pretty sure that is 3.2Mbps , so would 250Kbps come down to 7812,5 ? it is such a weird number.. Anyway i am sure you know better.

[Makuna]

That number looks close. It will be easier in a few days (hopefully).
The x8 and x16 encode channels into a bit of a datum that all channels are in. More complex. And with the new 3 step cadence will add some more code to enhance.

I am in a middle of some refactoring for ESP32 that reduces some duplication between i2S and thew new (still not in an official release) ESP32-S3 LCD parallel feature. Lots of testing and I found a problem with the i2s x16 encoding.

One of the things that got refactored was the timing. The new timing classes look like this...

class NeoBitsSpeedWs2812x
{
public:
    const static uint16_t BitSendTimeNs = 1250;
    const static uint16_t ResetTimeUs = 300;
};

So, you now set the total time in nanoseconds a single bit should take. Much easier to understand and modify.
Another thing with these changes is that 3 step and 4 step cadences will be supported. You can pick from either 4 slices of that time to produce the high pulse and low pulse (the old way) or 3 step cadence with 3 slices (the new way). 3 step saves memory.

[devarishi7]

Oh that does sound interesting, and a bit easier to calculate. I suppose that the 3 step or 4 step choice would apply to all the parallel channels, or can you pick per per channel ? I mean that is what is so different about the DMX512 / WS2821, which is basically 1 step, but with added start bit & stop bits. If 3 or 4 step becomes a choice per channel, then the 1 step would also be a simple choice.
As it is with NeoEsp8266I2sDmx512MethodBase, it just has a completely different FillBuffers() function to any of the other single data pin methods.

It is great to hear that memory reduction for the parallel methods is on the list. It was one of the reasons holding me back from using it. Generally i would want to have 4 universes (680 pixels) per channel, but already at 8 channels things were getting a bit out of hand, memory wise. A 25% reduction would be a serious improvement.

My request would initially be for the single channel method, which i think would be less complex anyway, a sort of copy paste from what you did for the esp8266. On my current project i added DMX out, but the only DMX library i found has some compatibility issues, and requires almost undivided attention from the core to function properly. The ESP32 UART and the FIFO is to complicated for me to understand, To be able to send a DMX512 universe in the background on one of the I2S channels while i send 4 WS2812 universes on the other seems like a lot more efficient solution.

If i ever get to a point of selling significant amount of product (oh i am such a dreamer.. ;-) ) a small donation per piece does make quite a lot of sense.

On the use of the esp8266 DMX512 i am still at only 10 pieces. You know i sell the hardware for a specific purpose and give the software away for free as a part of it.

[Makuna]

You won't be able to mix cadence nor speed on the same parallel peripheral. The first one sets the timing on the hardware and the hardware can only support one timing. The cadence change effects the speed (needs to send only 3 instead of 4 at the same time, runs slower to so).

[devarishi7]

That makes complete sense but how are you combining 800KHz & 400KHz methods and others with different speeds atm ? Just curious really, for me personally i don't think it will matter much. There are 2 I2S busses available. As for practical uses of that many channels i have difficulty thinking of them, but that is a different matter. I think for me just the single channel DMX output would be fine, although the practical use for a multi-channel DMX output exists.

[Makuna]

Can't combine 400 & 800, would have use separate peripherals.
400 is almost unheard of anymore. But you could replace the 400 with APA106 and the topic is still valid.

[devarishi7]

Can't combine 400 & 800, would have use separate peripherals. 400 is almost unheard of anymore. But you could replace the 400 with APA106 and the topic is still valid.

Yeah whatever, it was meant more as an example, i am just curious on the matter. Combining different output signals on a single device is anyway quite rare.

[Makuna]

#828
Those changes I was mentioned are now in Master.

[devarishi7]

#828 Those changes I was mentioned are now in Master.

I will have a look. Still aiming for DMX512 support for the ESP32 I2S.

[devarishi7]

Hmm OK i've had a look and, well great work ! of course. With the addition of the 3 cadence it would even make it a tad easier to add the 11 bit per Byte encoding, though i think not without a modification to the I2SInit()

i2sInit(_bus.I2sBusNumber, 
            false,
            2, // bytes per sample 
            T_CADENCE::DmaBitsPerPixelBit,
            T_SPEED::BitSendTimeNs,
            I2S_CHAN_STEREO, 
            I2S_FIFO_16BIT_DUAL, 
            dmaBlockCount,
            _i2sBuffer,
            _i2sBufferSize);

bits per pixelbit just doesn't cover that option somehow.
Edit: Maybe instead the number of bits per PixelByte would solve that
Of course adding a completely separate way like for the ESP8266 is also an option i guess.

Anyway those are just some thoughts.

[Makuna]

Modifying those core i2s functions is not a problem. Those layers are on a future work item to replace completely with newer APIs and a better design to gain some cross "version" (S3 and beyond) support. The Esp8266 layer is better designed.

[devarishi7]

Yes i only found 3 instances where that particular variable (DmaBitsPerPixelBit)was being used, and only to calculate either sample speed or output buffer size (i think) so changing that to Bits per Byte should be a small modification, after that the DMX512 method will be just a simple addition really (for you, that is, for me it would still be struggle) Basically it will be just using a different cadence, and with that a different encoder.

The Esp8266 layer is better designed.

With only 1 bus, and 1 architecture it is also a lot simpler i guess. The ESP32 provides functionality for different output on different busses and various architecture. The DMX512 method is in that case not fully incorporated into the methods but kept separate for the different filling of buffers it requires. (btw this must be a typo in the comments)
// output stream is uint31_t from NeoEsp8266I2sDmx512Method.h line 239
I keep getting confused by it every time i see it :-D

A little hard to tell for me what is a better design or not. Quite a lot of it is beyond me you know.

[devarishi7]

So i had some time to have a look and test the cadence 3 & 4 step. It seems to work both, though i started out with the (at the time) un-released version 2.8.0, and i wasn't quite sure what to do on how to choose between the cadence, but anyway i am sure it does work perfectly, at least on the WS2812b's i've tested it on. Now the T_CADENCE opens the door for adding a different encoding and since for DMX512 an 11 step encoding is required (1 start + 8 data + 2 stop) as you know i thought i'd have a go and see if some small modifications i could get closer towards that.

I had mentioned already that it would require to change the passing of the number of dmaBitPerDataBit to a dmaBitPerDataByte value, so i decided to see if i could manage to do so without breaking anything :-D
in ESP32_i2s.h i changed the prototype definition for

void i2sInit(uint8_t bus_num, 
    bool parallel_mode,
    size_t bytes_per_sample,
    //uint16_t dmaBitPerDataBit,
	uint16_t dmaBitPerDataByte,   //&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
    uint16_t nsBitSendTime,
    i2s_tx_chan_mod_t chan_mod, 
    i2s_tx_fifo_mod_t fifo_mod, 
    size_t dma_count, 
    uint8_t* data,
    size_t dataSize);

in ESP32_i2s.c i changed the prototype definition for

esp_err_t i2sSetSampleRate(uint8_t bus_num,
    //uint16_t dmaBitPerDataBit,
	uint16_t dmaBitPerDataByte,
    uint16_t nsBitSendTime,
    bool parallel_mode,
    size_t bytesPerSample);

and the function

void i2sInit(uint8_t bus_num, 
        bool parallel_mode,
        size_t bytesPerSample, 
        //uint16_t dmaBitPerDataBit,
		uint16_t dmaBitPerDataByte,  //&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
        uint16_t nsBitSendTime,
        i2s_tx_chan_mod_t chan_mod, 
        i2s_tx_fifo_mod_t fifo_mod, 
        size_t dma_count, 
        uint8_t* data, 
        size_t dataSize)

to mathc the prototype and where it calls

    i2sSetSampleRate(bus_num, 
            //dmaBitPerDataBit,
			dmaBitPerDataByte,  //&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
            nsBitSendTime, 
            parallel_mode, 
            bytesPerSample);

and within that function where it uses that variable to calculate the clock divider (i think

    //&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
    //double clkmdiv = (double)nsBitSendTime / bytesPerSample / dmaBitPerDataBit / bck / 1000.0 * I2sClkMhz * clkSampleAdj;
	double clkmdiv = (double)nsBitSendTime / bytesPerSample / dmaBitPerDataByte / bck / 1000.0 * I2sClkMhz * clkSampleAdj * 8;

Now all i had to do was change the 2 existing Cadence classes in NeoEsp32I2sMethod.h

class NeoEsp32I2sCadence3Step
{
public:
    //const static size_t DmaBitsPerPixelBit = 3; // 3 step cadence, matches encoding
	const static size_t DmaBitsPerPixelByte = 24; // 3 step cadence, matches encoding

&

class NeoEsp32I2sCadence4Step
{
public:
    //const static size_t DmaBitsPerPixelBit = 4; // 4 step cadence, matches encoding
    const static size_t DmaBitsPerPixelByte = 32; // 4 step cadence, matches encoding   //&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&

and the call to

    void Initialize()
    {
        size_t dmaBlockCount = (_i2sBufferSize + I2S_DMA_MAX_DATA_LEN - 1) / I2S_DMA_MAX_DATA_LEN;

        i2sInit(_bus.I2sBusNumber, 
            false,
            2, // bytes per sample 
            //T_CADENCE::DmaBitsPerPixelBit,
			T_CADENCE::DmaBitsPerPixelByte,   //&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
            T_SPEED::BitSendTimeNs,
            I2S_CHAN_STEREO, 
            I2S_FIFO_16BIT_DUAL, 
            dmaBlockCount,
            _i2sBuffer,
            _i2sBufferSize);

as well as the construct which uses the variable

    void construct(uint16_t pixelCount, size_t pixelSize, size_t settingsSize) 
    {
        _data = static_cast<uint8_t*>(malloc(_sizeData));
        // data cleared later in Begin()

        // must have a 4 byte aligned buffer for i2s
        // since the reset/silence at the end is used for looping
        // it also needs to 4 byte aligned
        /*size_t dmaSettingsSize = T_CADENCE::DmaBitsPerPixelBit * settingsSize;
        size_t dmaPixelSize = T_CADENCE::DmaBitsPerPixelBit * pixelSize;
        size_t resetSize = NeoUtil::RoundUp(T_CADENCE::DmaBitsPerPixelBit * T_SPEED::ResetTimeUs / (T_SPEED::BitSendTimeNs * 8 / 1000), 4);*/
		size_t dmaSettingsSize = T_CADENCE::DmaBitsPerPixelByte * settingsSize / 8;
        size_t dmaPixelSize = T_CADENCE::DmaBitsPerPixelByte * pixelSize / 8;
        size_t resetSize = NeoUtil::RoundUp(T_CADENCE::DmaBitsPerPixelByte * T_SPEED::ResetTimeUs / (T_SPEED::BitSendTimeNs * 8 / 1000) / 8, 4);

As you can tell i've been marking the places where i have made changes.
So far so good, all still works :-D

Now i have started on the having a go at the implementation of the DMX512. I've had a good look and though some confusion on certain matters i have something that at least compiles and in my opinion should work.

class NeoEsp32I2sCadence11Step
{
public: 
    const static size_t DmaBitsPerPixelByte = 11;
	
	static void EncodeIntoDma(uint8_t* dmaBuffer, const uint8_t* data, size_t sizeData)
    {
		uint16_t* pDma = reinterpret_cast<uint16_t*>(dmaBuffer);
        uint16_t dmaValue = 0;
        const uint8_t* pSrc = data;
        const uint8_t* pEnd = pSrc + sizeData;
		const size_t BitsInSample = sizeof(uint16_t) * 8;
		
		uint8_t destBitsLeft = BitsInSample;
		

	    *(pDma++) = 0x0000; // start the break.
		*(pDma++) = 0x0007; // end the break.  Breaklength = 16 + 11 * 4 = 108us BreakMab  3 * 4 = 12us
		

        while (pSrc < pEnd)
        {
			uint8_t dmxValue = *pSrc; 
			if (pSrc == data) // make sure channel 0 = 0 for DMX
			{
				dmxValue = 0;
			}
			
			if (!destBitsLeft)
			{
				destBitsLeft = BitsInSample;
				*(pDma++) = dmaValue;
				dmaValue = 0;
			}
			dmaValue = dmaValue << 1;  // start Bit
			dmaValue |= 1;
			destBitsLeft--;
						
			for (uint8_t i = 0; i < 8; i++) // data bits
			{
				if (!destBitsLeft)
				{
					destBitsLeft = BitsInSample;
					*(pDma++) = dmaValue;
					dmaValue = 0;
				}
				dmaValue = dmaValue << 1;
				dmaValue |= dmxValue & 1;
				dmxValue = dmxValue >> 1;
				destBitsLeft--;
			}
			
			for (uint8_t i = 0; i < 2; i++) // stop bits
			{
				if (!destBitsLeft)
				{
					destBitsLeft = BitsInSample;
					*(pDma++) = dmaValue;
					dmaValue = 0;   // not sure this is required, 16 bitshifts results and all bits being fresh.
				}
				dmaValue = dmaValue << 1;
				destBitsLeft--;
			}			
		}
	}
};

There is a few things of course. The break, i kept it simple and since the inverted bit is not part of the encoding like on the esp8266, i think i got it right. If it's actual DMX hardware that is being used instead of the ws2821 ledstrip, it anyway doesn't matter much, since one can easily swap the A & B lines of the transceiver.
Then there is the inversion of the bit order, which i figured would just as well be done in the process of the encoding rather than the whole data byte first. And adding the start & stop bits.

There were some unknowns to me in the original code. Things i was just unsure of particularly in the construct
dmaSettingsSize what is it for ?
resetSize probably ends up being both the break as well as the marker. I haven't started on the Speed thing yet, maybe looking at it will clarify things for me.

and i got seriously confused by the 'const' in const uint8_t* pSrc = data; until i decided that the const refers to what it is pointing to, not the pointer itself. Anyway if you share your thoughts. For me it was enough for today.

[devarishi7]

Ah i forgo this bit

		dmaValue = dmaValue << destBitsLeft;  // shift the significant bits fully left
		*(pDma++) = dmaValue;  // and store the remaining bits

And i am a little puzzeld about the braces around the pDma++ is there any effect ?

[devarishi7]

So I added the SpeedBase and the typedefs and i think i got those more or less correct.

class NeoBitsSpeedDmx512
{
public:
    const static uint16_t BitSendTimeNs = 4000;  // 4us, 250Kbps
    const static uint16_t ResetTimeUs = 128;  // set for 32 bits, include Break MAB
};

and

typedef NeoEsp32I2sMethodBase<NeoBitsSpeedDmx512, NeoEsp32I2sBusZero, NeoBitsNotInverted, NeoEsp32I2sCadence11Step> NeoEsp32I2s0Dmx512Method; 
typedef NeoEsp32I2sMethodBase<NeoBitsSpeedDmx512, NeoEsp32I2sBusZero, NeoBitsInverted, NeoEsp32I2sCadence11Step> NeoEsp32I2s0Dmx512InvertedMethod; 
typedef NeoEsp32I2sMethodBase<NeoBitsSpeedDmx512, NeoEsp32I2sBusOne, NeoBitsNotInverted, NeoEsp32I2sCadence11Step> NeoEsp32I2s1Dmx512Method; 
typedef NeoEsp32I2sMethodBase<NeoBitsSpeedDmx512, NeoEsp32I2sBusOne, NeoBitsInverted, NeoEsp32I2sCadence11Step> NeoEsp32I2s1Dmx512InvertedMethod; 

It all compiled, but then as expected it didn't work. I had the start & stop bits inverted, corrected that. and i got some result, but not what i wanted. The break was recognized ok. But somehow the data stream is not arriving properly, and i concluded that it isn't sent properly.
Now i stuck to the 16-bit format that you were using for the output stream, then tried an 8-bit and a 32-bit. Somehow i am quite sure i am missing some information about how the I2S buffer is being send. I tried making use of your code for the esp8266, but although i did get it to compile, did not yield any results. I know your code should be faster, but regardless of that i think my code should also just work. Did i just get the speed wrong somehow ? See for

class NeoBitsSpeedWs2812x
{
public:
    const static uint16_t BitSendTimeNs = 1250;
    const static uint16_t ResetTimeUs = 300;
};

That is just 800KHz regardless of the cadenze. Hmm any chance you are willing to help me out here. It is your code, you know your way around.

[Makuna]

I will take a look when I can. I am pretty busy so don't expect a quick reply.

Do you have a branch public on your own github of the current state you have?
If not, an alternative is to create a pull request with the title FOR REVIEW ONLY and that will give me insight into your total changes.

[devarishi7]

Oh i figured it out ! I had the epifanie this morning while lying in bed. The timing was off !
class NeoBitsSpeedDmx512

{
public:
    const static uint16_t BitSendTimeNs = 5500;  //   4us, 250Kbps so 4000 * 11 / 8
    const static uint16_t ResetTimeUs = 128;  // set for 32 bits, include Break MAB
};

The way the timing is set defines the BitSendTimeNs for a data bit. It should of course, but to get the proper speed for a format that just adds 3 bits extra to 8 bytes, it ends up coming down to multiplying it by 11 and dividing by 8 to compensate for the calculation done with those values. Anyway it does work now, but i will need to check if the reset time adds the proper amount of bytes to the buffer, i don't want to overrun it.

I will post the changes in a pull request so you can have a look.

It looks a bit strange like this, but of course it is a kind of hack into the code you wrote. Considering now to 'add' T_CADENCE::DmaBitsPerPixelByte rather than it replacing T_CADENCE::DmaBitsPerPixelBit
But so happy it works now. The details or just that. details.

[devarishi7]

Ok so Github is not something i use very often and the details on creating forks & branches and pull requests elude me a little. Maybe when i find so extra time and motivation on the matter. I found a more elegant way to be able to use the correct BitSendTimeNs and it actually requires less changes to files. Both Esp32_i2s.h & Esp32_i2s.c stay the same. NeoEsp32I2sMethod.h has the 11 step cadence added as well as changing DmaBitsPerPixelBit to DmaBitsPerPixelByte for all 3 cadences. The call to i2sInit() now divides those bits per byte by 8 to get the bits per bit, and a similar thing for resetSize. The dmaSettingsSize & dmaPixelSize are divided by 8 after the multiplication.
NeoEsp32I2sMethod + neobits.zip
I've added the modified files (sorry for cutting some corners like that) and to get a default RGB sequence in a sketch i use the NeoBrgFeature which sets ch 1 to R and so forth, and use 171 pixels for a complete universe.
On occasion i have a tiny glitch but i am quite sure it is a hardware issue. I am working on a breadboard still, and i ended up moving it quite a lot during testing.
I figure it will be easy enough to add a NeoChannelFeature (or DmxChannelFeature) which is just a single byte in size, which will make it a bit more conforming for what it's purpose, but this works like a charm, and usually i end up writing straight to buffer from within my program anyway. If you have time and if you are curious enough, have a look. Thank you so much anyway for this amazing library !!

[devarishi7]

Ok i will have to sort out how to use github properly. I changed a few small things to make channel 0 part of the break so ch 1, 2 3 can be RGB to the same pixel, but that had some more consequences. I also suspect now that the glitch i saw was actually caused by something else. Although there is filling of the last 16-bit word with '1' 's the buffer is 32-bit aligned, which may result in there being a a whole 16-bits of '0' , which is enough to be recognized as a 'false' break. With the break time now being more than 4 bytes, this may be even worse. The break was a perfect 4 bytes before and now ends up being 6 bytes, so chances are that the last 2 bytes will be empty and the 2 bytes before that wont be. And those last few bytes of the buffer are looping right ? They should be either all HIGH or all LOW. Best is all HIGH i guess, so i figured something like this for the construct

		if (T_CADENCE::DmaBitsPerPixelByte == 11) // for DMX we set bits all HIGH
		{
			memset(_i2sBuffer, 0xff, _i2sBufferSize);  //
		}
		else
		{
			memset(_i2sBuffer, 0x00, _i2sBufferSize);
		}

I will work out the branch and fork thing properly, but that will take some time. I also have an idea on how to significantly speed up the 3 step encoding, which should make it nearly as fast as the 4 step.

[devarishi7]

Ok so i've finally managed to create the pull-request, hope i did it all correctly o OIt was a bit of a challenge. I have also added the 3-step faster encoding, and tests have shown that to work without any issue. As for the DMX512 encoding, i am pretty sure that i've gotten the encoding and the timing correct, but i am still getting glitches, sometimes every few thousand frames or so. Basically rarely but undeniable when watching the processed from a receiver. I don't have an oscilloscope to try and figure out what is wrong. What i have excluded is

• hardware issue (same hardware using a UART method works perfectly)
• Timing issue (the issue is not more dominant when listening to addresses further down the line)
  I have fiddled with the break length, even went as far as doing the whole thing inverted.

What confuses me a little is that it is required to put the break at the start of the stream. I was under the impression that the last section of the data-stream gets looped until a new stream starts, which would basically mean we could start with just the MAB, if the last few words of the stream are 0x0. If the output pin is just idle at the end that would make sense, but it appears that it just isn't, which may be the cause of the occasional glitch. I also have no explanation for the WS2812b working without any glitch, but i am confident that the internal workings on the I2S bus are beyond me, and also that the reason / cause is there. I might be wrong though.

I also tried with a pullup (and with a pulldown) resistor on the output pin, just in case it is cause by the pin being temporarily in input mode for some reason, but no luck. Anyway if you have time, have a proper look.