Another audio visualizer

[Nitin-Arora82]

Hello,

First off, many thanks for putting in the effort and making this available publicly. Can't thank the contributors enough. As the subject line describes, I am trying to put together an audio visualizer (64x64 single HUB75 panel). I had made this earlier using MAX7219 modules and MD Parola library so a lot of the code is carried over. The sketch works fine other than the part where I try to draw the bars on the panel. With the Parola library I was able to draw a frame in the background and then update the display manually. I understand that this is not how this library works. I have tried a few different approaches but have failed miserably.

My broken code is as follows and I appreciate that it is untidy at the moment but I am trying to solve one problem at a time. In essence the problem is that while the display updates with data from the barHeights array, it is very "choppy". I am not sure how else to describe this. What I mean to say is that the display updates but in discrete frames which looks very laggy and out of sync with the audio.

Interestingly if I replace the following line:
barHeights[i/divisor] = log(fftValue + 1)/log(255) * 64;

with the following to simulate random values, the behavior is fluid:
barHeights[i/divisor] = random(1, 64);

I know I am doing something foolish, I just can't seem to figure out what it is. Would really appreciate some guidance on this. My code (I have excluded some parts that ideally should not be relevant to the issue that I am seeking help with):

//MatrixPanel_I2S_DMA dma_display;
MatrixPanel_I2S_DMA *dma_display = nullptr;

uint8_t numBands;
uint8_t barWidth;
uint8_t pattern;
uint8_t brightness;
uint8_t currentBrightness;
uint16_t displayTime;
bool autoChangePatterns = false;
bool inverted = false;

#include "audio_reactive.h"
#include "web_server.h"


uint16_t peak[] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 };
uint16_t prevFFTValue[] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 };
uint16_t barHeights[] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 };


void setup() {
  Serial.begin(115200);
  //Serial.println ("Setting up audio");
  setupWebServer();
  setupAudio();
  currentBrightness = 150;
  brightness = 150;
  numBands = 64;
  pattern = 0;

  HUB75_I2S_CFG mxconfig(
    PANEL_RES_X,  // module width
    PANEL_RES_Y,  // module height
    PANEL_CHAIN   // Chain length
  );

  dma_display = new MatrixPanel_I2S_DMA(mxconfig);
  dma_display->begin();
  dma_display->setBrightness8(currentBrightness);  //0-255
  dma_display->clearScreen();
}

void loop() {

  uint8_t fftValue;
  uint8_t divisor = 1;

  divisor = 64 / numBands;

  if (brightness != currentBrightness) {
    dma_display->setBrightness(min((int)brightness, 255));
    currentBrightness = min((int)brightness, 255);
  }
  for (int i = 0; i < (numBands * divisor); i += divisor) {
    if (numBands == 32) fftValue = (fftResult[i] + fftResult[i + 1]) / 2;
    else if (numBands == 16) fftValue = (fftResult[i] + fftResult[i + 1] + fftResult[i + 2] + fftResult[i + 3]) / 4;
    else fftValue = fftResult[i];

    fftValue = ((prevFFTValue[i / divisor] * 3) + fftValue) / 4;

    barHeights[i / divisor] = log(fftValue + 1) / log(255) * 64;  // Scale bar height
    //barHeights[i/divisor] = random(1, 64);                  // Random junk data

    prevFFTValue[i / divisor] = fftValue;

    if (barHeights[i / divisor] > peak[i / divisor])  // Move peak up
      peak[i / divisor] = min(32, (int)barHeights[i / divisor]);
  }
  switch (pattern) {
    case 0:
      delay(15);
      dma_display->clearScreen();
      for (int x = 0; x < (numBands * divisor); x += divisor) {
        for (int y = 0; y < barHeights[x / divisor]; y++) {  //Draw bars non-inverted
          dma_display->drawPixelRGB888(x, y, 255, 0, 0);
        } 
      }
      break;
    case 1:
      delay(15);
      dma_display->clearScreen();
      for (int x = 0; x < (numBands * divisor); x += divisor) {
        for (int y = 0; y < barHeights[x / divisor]; y++) {  //Draw bars non-inverted
          dma_display->drawPixelRGB888(x, y, 0, 255, 0);
        }
      }
    break;
    default:
      break;
  }
  /*if (pattern != 2 && pattern != 3 && pattern != 4){
    for (int p = 0; p < (numBands * divisor); p+=divisor){                          //Draw peaks only if pattern is not center-bars
      //dma_display->drawLine(peak[p / divisor], p, (p + divisor - 1), dma_display->color444(15, 0, 0));
    }
   }
  EVERY_N_MILLISECONDS(55){                                                   //Decay Peaks
      for (uint8_t band = 0; band < numBands; band++)
      if (peak[band] > 0) peak[band] -= 1;  
  }*/
}

[board707]

Interestingly if I replace the following line:
barHeights[i/divisor] = log(fftValue + 1)/log(255) * 64;

with the following to simulate random values, the behavior is fluid:

barHeights[i/divisor] = random(1, 64);

If this is not a typo, it means that esp32 evaluates the first expression too long.
Let's try to simplify it and convert it to integer mathematics:

// calculate the log2 of fftValue + 1
uint16_t x = fftValue + 1; 
uint8_t r =0;  // r will be log base 2 of (x)

while ( x >>= 1)  {
 r++;
}
barHeights[i/divisor] = r/8 * 64;

[Nitin-Arora82]

Thank you so much for replying here. Your suggestion got me thinking.

I will elaborate a bit here for anyone else who might have related, but not the same problem.

I had thought about this calculation taking too long and consequently making the display laggy. However, I had included both the lines in a sketch (leaving the final output as the random numbers) and behavior was as expected. I assumed that I had a problem elsewhere. Clearly I should have included this information in my initial post, so apologies.

What you mentioned had me look through my FFT code again since it seemed like the frame data itself was arriving late. And there it was - 3 sneaky Serial.println() statements that were holding up the FFT code.

Removed those and now the thing works like a charm. Forward and onwards!!

Many thanks again!!

[Nitin-Arora82]

Resolved as mentioned above