3.0.0-beta1 | Open Beta!

Sensory Bridge Firmware 3.0.0 is coming soon, and I'd like early tester feedback!

---

(Aside from what's detailed in this release text, the new changes are covered in the code comments, but some comments are outdated as of late, or still a work in progress. This update is intended for interested enthusiasts familiar with the Arduino IDE.)

The biggest change of the 3.0.0 milestone is the removal of FFT-based frequency detection, instead opting for a parallelized Goertzel-based approach that I'm calling "GDFT".

That acronym can mean one of two things:

• "Goertzel-based Discrete Fourier Transform", or my favorite:
• "God Damn Fast Transform"

❤️

It's true, this is hugely faster (>100 FPS vs. ~70 FPS on 2.0.0), and only resolves the frequencies that Sensory Bridge is interested in. This means that your display is perfectly chromatic, with 64 notes displayed at exact intervals, which are now individually colored by default! An "E" note will always be the same color, no matter what octave it's in.

The new default frequency range of the display is 110Hz to 4186Hz, the upper 64 keys of a piano! However, it's now configurable. Read more about that in the UART Command Line section below.

The biggest benefit to GDFT though is in the low frequencies like basslines and kicks. They're actually discernable now, instead of being a blurry FFT mess! All notes are treated and displayed equally on 3.0.0.

New Features So Far

• GDFT Overhaul
• New spectrogram filtering techniques for smoother AND faster output
• Runtime configuration of many variables via the expanded UART menu such as LED protocol type (Neopixel/Dotstar)

Limitations

This is an incomplete beta version, and there are definitely missing features. Still missing are:

• Sweet Spot LEDs
• Noise Calibration feedback animation
• All modes except GDFT (All the other modes were derived from FFT and still need porting)
• Chroma Knob functionality

Coming Soon

New features still in the queue for 3.0.0 are:

• Color Themes
• Microphone frequency response calibration routine

Expanded UART Menu

The UART menu from 2.0.0 is still here, and contains a LOT more to do now. Here's the output if you type "help":

SENSORY BRIDGE - Serial Menu ---------------------------------------

                                   v | Print firmware version number
                               reset | Reboot Sensory Bridge
                       factory_reset | Delete all configurations, reboot
                       get_main_unit | Print if this unit is set to MAIN for SensorySync
                                dump | Print tons of useful variables in realtime
                                stop | Stops the output of any enabled streams
                                 fps | Return the average FPS

                       stream=[type] | Stream live data to a Serial Plotter.
                                       Options are: audio, fps, max_mags, max_mags_followers, magnitudes, spectrogram
     led_type=['neopixel'/'dotstar'] | Sets which LED protocol to use, 3 wire or 4 wire
        led_count=[int or 'default'] | Sets how many LEDs your display will use (native resolution is 128)
                        debug=[bool] | Enables debug mode, where functions are timed
       sample_rate=[hz or 'default'] | Sets the microphone sample rate
     note_offset=[0-32 or 'default'] | Sets the lowest note, as a positive offset from A1 (55.0Hz)
      square_iter=[int or 'default'] | Sets the number of times the LED output is squared (contrast)
  magnitude_floor=[int or 'default'] | Sets minimum magnitude a frequency bin must have to contribute the show
   max_block_size=[int or 'default'] | Sets the maximum number of samples used to compute frequency data
samples_per_chunk=[int or 'default'] | Sets the number of samples collected every frame
           gain=[float or 'default'] | Sets the scaling of spectrogram data (>0.0 is brighter, <0.0 is darker)
                set_main_unit=[bool] | Sets if this unit is MAIN or not for SensorySync

Example usage: note_offset=12 (You might need the newline character enabled in Serial Monitor)

Just so you're aware, these menu input are not sanitized yet. You can enter invalid settings and it will blindly trust you, possibly leading to a boot loop, requiring you to erase the chip and reflash. No "safe mode" yet either.

Update Process

To update your Sensory Bridge firmware to the 3.0.0-beta1, download the "Source code (zip)" at the bottom of this page, extract it, and open the "SENSORY_BRIDGE_FIRMWARE.ino" in Arduino IDE.

If you don't yet have ESP32-S2 in your Arduino Board Definitions, follow the steps on this page:
https://learn.adafruit.com/adafruit-metro-esp32-s2/arduino-ide-setup-2

Install FastLED Dependency

If you don't have it installed in your copy of Arduino yet, you'll need it for Sensory Bridge to compile.

• Go to Sketch > Include Library > Manage Libraries
• In the window that appears, search for "FastLED" (by Daniel Garcia)
• Install the latest version

Board Settings

Finally, make sure that all of your board settings match this image:

Your Serial port (i.e. "COM10") will likely differ from mine.

IMPORTANT PLEASE READ

To use this beta firmware, you need to make sure the following setting is enabled in Arduino:

Erase All Flash Before Sketch Upload: ENABLED

This will erase your 2.0.0 config.bin and noise_cal.bin files in your Sensory Bridge's storage, because they are incompatible with this version and can cause your device to not boot correctly. A more graceful solution is in the works to detect config file versions and refuse incompatible ones.

"USB CDC On Boot: Enabled" is quite important, as this is what will allow you to flash firmware over USB again next time and use the UART menu. If this is not enabled, the next time you want to update your firmware you'll need to hold down the BOOT button on the bottom of your Sensory Bridge while plugging it in to be able to flash it again as normal.

Arduino IDE 2.x Fixes

Unfortunately, the new Arduino IDE 2.x has some issues that I'm hoping are fixed in a future update, because they can break this updating process and all of the new features are weighed down by annoying changes from the Legacy IDE. Here's how to fix those:

DISABLE DEFAULT COMPILATION WARNING BEHAVIOR

(This will make the firmware actually usable)

• In the Arduino 2.x IDE, click "File > Preferences"
• Set "Compiler Warnings" to "Default"
• This is because some external dependencies in the code such as FastLED trigger ignorable warnings
• Arduino IDE 2.x treats all warnings as errors by default. (WTF)

DISABLE WORD WRAPPING

(This will guarantee code and docs are more readable, it's 2023, 80-column monitors aren't a thing. No IDE needs word wrapping enabled by default, that's dumb.)

• In the Arduino 2.x IDE, hit F1
• In the textbox that appears, type "wrap"
• You'll see a result for "View - Toggle Word Wrap"
• Click this item
• Repeat at least once until buggy word-wrapping is actually disabled

2.0.0 | Full rewrite, tons of new features!

Sensory Bridge Firmware 2.0.0 is ready! ❤️

Almost all code has been rewritten from scratch. In the development process from March 2022 to the release in September, the original firmware files became spaghetti code full of unused variables and strange redundancies. Rather than try to documenting that code, I've studied it and started over.

The new FFT post-processing is so much quicker that the output can be delayed 3 frames without any noticeable lag. This allows for a look-ahead approach to reducing flicker/noise in the display.

You can sync multiple units now. There's new modes to enjoy. The Boost knob now controls color. Noise calibration finds any DC offset in the mic, and finds a noise floor for the waveform data too, and more. Be sure to read the full changelog below to learn how these new features work!

Update Process

To update your Sensory Bridge firmware to 2.0.0, download the "Source code (zip)" at the bottom of this page, extract it, and open the "SENSORY_BRIDGE_FIRMWARE.ino" in Arduino IDE.

If you don't yet have ESP32-S2 in your Arduino Board Definitions, follow the steps on this page:
https://learn.adafruit.com/adafruit-metro-esp32-s2/arduino-ide-setup-2

Install FastLED Dependency

If you don't have it installed in your copy of Arduino yet, you'll need it for Sensory Bridge to compile.

• Go to Sketch > Include Library > Manage Libraries
• In the window that appears, search for "FastLED" (by Daniel Garcia)
• Install the latest version

Board Settings

Finally, make sure that all of your board settings match this image:

Your Serial port (i.e. "COM10") will likely differ from mine.

"USB CDC On Boot: Enabled" is quite important, as this is what will allow you to flash firmware over USB again next time. If this is not enabled, the next time you want to update your firmware you'll need to hold down the BOOT button on the bottom of your Sensory Bridge while plugging it in to be able to flash it again as normal.

Arduino IDE 2.0 Fixes

Unfortunately, the new Arduino IDE 2.0 has some issues that I'm hoping are fixed in a future update, because they can break this updating process. Here's how to fix those:

DISABLE DEFAULT COMPILATION WARNING BEHAVIOR

(This will make the firmware actually usable)

• In the Arduino 2.0.x IDE, click "File > Preferences"
• Set "Compiler Warnings" to "Default"
• This is because some external dependencies in the code trigger ignorable warnings
• Arduino IDE 2.x treats all warnings as errors by default.

DISABLE WORD WRAPPING

(This will guarantee code and docs are more readable)

• In the Arduino 2.x IDE, hit F1
• In the textbox that appears, type "wrap"
• You'll see a result for "View - Toggle Word Wrap"
• Click this item
• Repeat at least once until buggy word-wrapping is actually disabled

2.0.0 Changelog

Only the largest/most relevant changes are here, as this was a full re-write!

FFT MODE

• Uses minor frame delay to allow for a look-ahead solution to noise/flicker reduction
• FFT image is now blurred with a 3px kernel
• Mood knob now affects FFT directly, not just the LED output of it, meaning Mood automatically affects FFT-derived modes

BLOOM MODE

• New "Fast" Bloom mode added, with 2x scroll speed

WAVEFORM Mode

• Contrast has been improved

VU MODE

• Completely re-written, is now derived from waveform instead of FFT data
• New "Dot Mode" with motion blur and antialiasing effects!

AUDIO CAPTURE

• I2S audio sample parser now uses more accurate math to correct the microphone DC offset
• A new noise calibration feature was added: automatic estimation of the noise floor in waveform data, for use in modes which don't have FFT features

AUTOMATIC BOOST

• The Boost Knob no longer controls audio sensitivity
• Sensory Bridge now automatically compensates for audio volume in a graceful fashion, without destroying any dynamic range of the final light show
• Instead, the Boost Knob now controls CHROMA:

CHROMA KNOB

• It still says "Boost" on Rev 1. PCBs, but that knob is now for color!
• The first 95% of that knob's turning radius will allow you to manually select a base hue for the lights and keep it that way
• If turned all the way up into the last 5% of the range, color returns to automatic mode, where hue cycling is based on musical content (original behavior)
• (This knob will be relabeled to "CHROMA" on future Rev. 1.1 PCBs)

SWEET SPOT

• Because the Sweet Spot LEDs are no longer tied to manual boost control, their behavior is different but their meaning stays the same.
• Sweet Spot now indicates if the audio source is too quiet (too close to noise floor to reliably calculate FFT even with new auto-ranging) or too loud (near audio clipping limit)

WIRELESS SYNCHRONIZATION

It's finally here! If you own more than one Sensory Bridge, flash this update to all of them, and you can get them synchronized!

Choose a Sensory Bridge that you want to act as a main controller for all others. Open a Serial Monitor to it at 500,000 baud, and type the following command:

set_main_unit=true

If the command was accepted, your unit should reboot, and any other units in the room should now be staying in sync with any changes you make on the main unit! Try changing the mode, starting a noise calibration, or making the color cycle by clapping loudly! Settings and physical control interactions are synchronized between units, but audio/relevant data is NEVER shared wirelessly. Each Sensory Bridge still does all of its own lightshow calculations.

You can always undo this change later with a "set_main_unit=false" command.

NOTE 1: If you try to modify the knobs or hit buttons on a unit that isn't the MAIN, it will alert you with a flash on both the unit you're touching, and the unit set as the MAIN!

NOTE 2: Currently, only one group of synched units is possible in a household, as all units within range of a powered on MAIN unit will synchronize indiscriminately.

This is a beta feature which is still being developed, let me know how it goes!

UART Command Line

Also in the works is a way to control deeper settings with an interactive prompt via UART, (500,000 baud) until a proper web-app is complete. Currently, it only accepts manually written commands:

• v - Print firmware version
• reboot - Reboot Sensory Bridge
• factory_reset - Clears all saved configurations
• get_main_unit - Returns whether this unit is set as MAIN or not
• set_main_unit={bool} - Set whether this unit is MAIN or not (i.e. "set_main_unit=true")

Questions / Feedback

If you have any questions/feedback about this new update, the Lixie Labs Twitter Page is the best place to reach me!

1.1.0 | First Fixes!

Sensory Bridge Firmware 1.1.0 is ready!

Several changes have been made, mostly to improve FFT mode, but things like button usage and Bloom Mode have been rewritten as well!

Update Process

To update your Sensory Bridge firmware to 1.1.0, download the "Source code (zip)" at the bottom of this page, extract it, and open the "SENSORY_BRIDGE_FIRMWARE.ino" in Arduino IDE.

If you don't yet have ESP32-S2 in your Arduino Board Definitions, follow the steps on this page:
https://learn.adafruit.com/adafruit-metro-esp32-s2/arduino-ide-setup-2

Install FastLED Dependency

If you don't have it installed in your copy of Arduino yet, you'll need it for Sensory Bridge to compile.

• Go to Sketch > Include Library > Manage Libraries
• In the window that appears, search for "FastLED" (by Daniel Garcia)
• Install the latest version

Board Settings

Finally, make sure that all of your board settings match this image:

"USB CDC On Boot: Enabled" is quite important, as this is what will allow you to flash firmware over USB again next time. If this is not enabled, the next time you want to update your firmware you'll need to hold down the BOOT button on the bottom of your Sensory Bridge while plugging it in to be able to flash it again as normal.

1.1.0 Changelog

FFT MODE

• FFT has been largely improved with better smoothing
• Now uses 4 zones for auto-ranging, reducing an issue where prominent frequencies could drown out the rest of the FFT
• Now has two modes: Linear and Log, which change how the FFT is scaled
• Inverted mode has been removed

BLOOM MODE

• Contrast has been improved
• Mirroring no longer scales Bloom Mode, keeping the fast scrolling
• Bloom Mode now has edges that gradually darken
• Color algorithm is now consistent with other modes

WAVEFORM Mode

• Contrast has been improved

AUDIO CAPTURE

• Captured audio is now slightly amplified to allow better range of the Boost Knob. "Sweet Spot" value was modified to match.

NOISE CALIBRATION

• An animation is now played during the debounce wait, instead of just freezing the last image
• Preview colors have changed

BUTTONS

• Buttons are now non-blocking, and long-presses no longer require releasing the buttons to register.
• Button definitions have moved to structs with access to press status and duration

1.0.0 | Initial Release

Update README.md