Coupling NEORV32 and AXI CDMA

[Sam-Vervaeck]

I have coupled the NEORV32 to an AXI CDMA and it works mostly except for some "busy waiting" behaviour which I cannot figure out.

My test-case was to copy some data from one BRAM to another:

I have populated BRAM 0 with a COE file and after copying data to BRAM 1 I check if the same values are in the same relative memory offsets. For this functionality, I adapted the following function to set the relevant CDMA registers. Towards the end you can see a "busy waiting" section that checks the status register to see if the idle bit has already been set:

// This function initiates a DMA transaction. DMA copies num_bytes many bytes from src_addr to
// dest_addr. Both addresses are AXI addresses. 
// The function blocks until completion if block != 0. Otherwise it returns immediately.
void neorv32_cdma_transfer(uint32_t dest_addr, uint32_t src_addr, uint32_t num_bytes, uint8_t block)
{
	uint32_t block_val;

	//set source address:
	if(DEBUG) neorv32_uart0_printf("CDMA: setting source address to 0x%x\n", src_addr);
	CUSTOM_CDMA->CDMA_SA_reg = src_addr;
	if(DEBUG) neorv32_uart0_printf("                             is 0x%x\n", CUSTOM_CDMA->CDMA_SA_reg);

	//set dest address:
	if(DEBUG) neorv32_uart0_printf("CDMA: setting destin address to 0x%x\n", dest_addr);
	CUSTOM_CDMA->CDMA_DA_reg = dest_addr;
	if(DEBUG) neorv32_uart0_printf("                             is 0x%x\n", CUSTOM_CDMA->CDMA_DA_reg);

	//set number of transferred bytes. This also starts the transfer:
	CUSTOM_CDMA->CDMA_BTT_reg = num_bytes;
	if(DEBUG) neorv32_uart0_printf("CDMA: sending %u bytes\n", num_bytes);

	if(!block)
		return;

	//wait for completion:
	if(DEBUG) neorv32_uart0_printf("CDMA: waiting for completion of the transfer...\n");
	do
	{
		block_val = CUSTOM_CDMA->CDMASR_reg;
	} while(!(block_val & (1<<1)));

	if(DEBUG) neorv32_uart0_printf("CDMA: done waiting\n");
}

The whole point of a CDMA is that the NEO can continue working on other instructions while the CDMA does its thing. However, sometimes you need to wait on the data to have done copying so that is what the block parameter is for. When I created test code with the block parameter set to 1 and checked the waves, the address of the idle bit does not get checked during the CDMA execution? Weirdly enough, it only gets checked once after the CDMA has already been done for some time:

On the screenshot you can see the reading signals in blue and the writing in red. In the middle you can see the CDMA doing its copying and on the location of my yellow time marker the idle bit gets checked for the first time at register offset 0x04. I would have expected to see more reads on the AXI4-Lite status connection of the CDMA during execution according to my transaction C code?

Anybody has any experience with an AXI CDMA or a similar design and knows what is going on?

[stnolting]

What is the base address of the AXI CDMA? Have you enabled any NEORV32 caches?

[Sam-Vervaeck]

I will check with my thesis supervisors if they are willing to look at it for some time as well

[Sam-Vervaeck]

Hey! I have a solution update. I got the busy waiting doing its thing after playing around with (mainly) the printf statements:

(the blue wave part at the top is the AXI4-Lite)

When turning these two printf statements off in between starting the CDMA and starting the busy waiting, it appears clearly on the previous waves screenshot:

        AXI_CDMA->CDMA_BTT_reg = num_bytes;
	// if(DEBUG) neorv32_uart0_printf("CDMA: sending %u bytes\n", num_bytes);

	if(!block)
		return;

	//wait for completion:
	// if(DEBUG) neorv32_uart0_printf("CDMA: waiting for completion of the transfer...\n");
	while (!(AXI_CDMA->CDMASR_reg & (1<<1)));

I created a larger CDMA transfer to time the duration of these prints and they seem to take 260 us:

So quite a bit longer than I had expected because that is 26000 clock cycles at 100 MHz if I am not mistaken. I suppose that has to do with the simulation of the UART (even though I am using the virtual sim one)?

[stnolting]

I suppose that has to do with the simulation of the UART (even though I am using the virtual sim one)?

Well, printf takes some time as all the formatting and especially the number conversion is quite compute intense.

However, in your code the printing is not part of the DMA polling loop:

while (!(AXI_CDMA->CDMASR_reg & (1<<1)));

So why should these printfs matter here? 🤔

[Sam-Vervaeck]

So why should these printfs matter here? 🤔

So the reason why on my first screenshot all the way at the beginning of this thread there was no "active polling" visible on the rvalid and rready wires is because it polls for the first time when the CDMA is already idle for a while. This delay was because of the two printf statements which made the polling start after the data transfer was already completed. The first poll would then immediately see an idle CDMA and the while loop would be stopped.

To see some actual active polling, I had to make sure it starts while the data transfer is still busy by taking out those two printf statements.

[stnolting]

Ahh ok, now I understand.
And yes, that makes sense.

Anyway, good to hear that the system works as expected now. 🚀