-
-
Notifications
You must be signed in to change notification settings - Fork 32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
beat-match script improvments #35
Comments
I have been trying to Find Kyle Dixon and/or convert that script to Node as well for this very purpose! I gave up after a couple of days and came here to suggest exactly this. |
I've been using music-beat-detector to accomplish this but it's limited to MP3 files and youtube videos. |
I think this is his github https://github.com/crocokyle |
I think the above is a good start, I just need to research how to pump alsa into |
I'll continue hacking away at it and ping you if I come up with anything |
Another (possibly false) start: import { MusicBeatDetector } from 'music-beat-detector'
import mic from 'mic-stream'
const musicBeatDetector = new MusicBeatDetector()
mic()
.pipe(musicBeatDetector.getAnalyzer())
.on('peak-detected', (pos, bpm) => {
console.log('PEAK', pos, bpm)
}) In my super-quick testing it only seemed to detect a single peak, and no bpm, but it may be the start of getting it to work. |
I was really impressed with how little code it took to start getting beat analysis with |
This is what I crammed together using some of the logic from the disco script and const Speaker = require("speaker");
const createMusicStream = require("create-music-stream");
const { MusicBeatDetector, MusicBeatScheduler, MusicGraph } = require(".");
const TPLSmartDevice = require("tplink-lightbulb");
const musicSource = process.argv[2]; //gets the first argument on cli
//MusicBeatScheduler syncs any detected peak with the listened audio. It's useful to control some bulbs or any other effect
const musicBeatScheduler = new MusicBeatScheduler(pos => {
new function() {
var self = this;
this.lamp = [
// list all of your lamps
"192.168.0.10"
];
this.init = function() {
for (var i in this.lamp) {
var light = new TPLSmartDevice(this.lamp[i]);
this.setLightColor(light);
}
};
this.setLightColor = function(light) {
var randomHue = parseInt(Math.random() * 360),
randomSaturation = parseInt(Math.random() * 100),
randomBrightness = Math.floor(Math.random() * (100 - 50 + 1) + 50);
light.power(true, 5, {
color_temp: 0,
mode: "normal",
hue: randomHue, // 0 to 360
saturation: randomSaturation, // 0 to 100
brightness: 100
});
};
}().init();
});
//MusicBeatDetector analyzes the music
const musicBeatDetector = new MusicBeatDetector({
sensitivity: 0.5,
scheduler: musicBeatScheduler.getScheduler()
});
//get any raw pcm_16le stream
createMusicStream(musicSource)
//pipe on analyzer
.pipe(musicBeatDetector.getAnalyzer())
.on("end", () => {
console.log("end");
})
//pipe on speaker
.pipe(new Speaker())
.on("open", () => musicBeatScheduler.start()); |
looks good! What if you swapped |
Yeah I've been messing with that, but keep getting this error when I use mic-stream
|
Okay I think that was an issue with the dependencies because I didn't have Sox installed. Will take another look this evening |
Okay I got |
Yeh, I think the original didn't pipe to the speakers. I'll be honest, pipes aren't my strong-suit. I often get lost in the pipes after a few throughs. Does it need to pipe to an output? |
It doesn't. That's been my hangup with it. Obviously, it's already playing on the speakers so it doesn't need to pipe to |
I'm starting to wonder if music-beat-detector doesn't work how I think it does. I tried out a nice portaudio-wrapper naudiodon: import { createWriteStream } from 'fs'
import { AudioIO, SampleFormat16Bit } from 'naudiodon'
// Create an instance of AudioIO with inOptions, which will return a ReadableStream
var ai = new AudioIO({
inOptions: {
channelCount: 2,
sampleFormat: SampleFormat16Bit,
sampleRate: 44100,
deviceId: -1 // Use -1 or omit the deviceId to select the default device
}
})
// Create a write stream to write out to a raw audio file
var ws = createWriteStream('rawAudio.raw')
//Start streaming
ai.pipe(ws)
ai.start() This worked fine and saved Then I tried to do same with music-beat-detector, with basically the same code: import { AudioIO, SampleFormat16Bit } from 'naudiodon'
import { MusicBeatDetector } from 'music-beat-detector'
import { createWriteStream } from 'fs'
// You can use this kind of stuff to select another device
// const devices = naudiodon.getDevices()
const ai = new AudioIO({
inOptions: {
channelCount: 2,
sampleFormat: SampleFormat16Bit,
sampleRate: 44100,
deviceId: -1 // Use -1 or omit the deviceId to select the default device
}
})
const musicBeatDetector = new MusicBeatDetector()
const analyzer = musicBeatDetector.getAnalyzer()
const ws = createWriteStream('rawAudio.raw')
ai.pipe(analyzer)
.on('peak-detected', (pos, bpm) => {
console.log('PEAK', pos, bpm)
})
.pipe(ws)
ai.start() It also writes On mac it exits immediately, and on linux it seems to keep recording, but I'm really not sure if it's working (no mic.) Without the |
I think you need to start the stream and then pipe it but don't take my word for it. I had to take the stream of the |
The above code is taken directly from the |
Maybe |
After piping to |
Yeh, I went back to music-beat-detector docs, and tried same idea: import { AudioIO, SampleFormat16Bit } from 'naudiodon'
import { MusicBeatDetector, MusicBeatScheduler } from 'music-beat-detector'
import { createWriteStream } from 'fs'
// You can use this kind of stuff to select another device
// const devices = naudiodon.getDevices()
const ai = new AudioIO({
inOptions: {
channelCount: 2,
sampleFormat: SampleFormat16Bit,
sampleRate: 44100,
deviceId: -1 // Use -1 or omit the deviceId to select the default device
}
})
const musicBeatScheduler = new MusicBeatScheduler(pos => {
console.log(`peak at ${pos}ms`) // your music effect goes here
})
const musicBeatDetector = new MusicBeatDetector({
scheduler: musicBeatScheduler.getScheduler()
})
const ws = createWriteStream('rawAudio.raw')
ai
.pipe(musicBeatDetector.getAnalyzer())
.on('peak-detected', (pos, bpm) => {
console.log('PEAK', pos, bpm)
})
.pipe(ws)
ai.start()
musicBeatScheduler.start() Again, on OSX it just exits right away, but after it does this (with some techno blaring):
Not sure what When I tested on linux (without mic) it stayed open & wrote to the file, but I'm really not sure if it's working (no mic.) I will see if I can find a mic somewhere. |
switching them did same: musicBeatScheduler.start()
ai.start() on mac it called my callbacks:
but then:
and then quit. |
Is it just exiting at the first beat? |
I tried combining the demo-code, and got the same problem: const fs = require('fs')
const { MusicBeatDetector, MusicBeatScheduler, MusicGraph } = require('music-beat-detector')
const { AudioIO, SampleFormat16Bit, getDevices } = require('naudiodon')
console.log(getDevices())
const musicGraph = new MusicGraph()
const ws = fs.createWriteStream('rawAudio.raw')
const ai = new AudioIO({
inOptions: {
channelCount: 2,
sampleFormat: SampleFormat16Bit,
sampleRate: 44100,
deviceId: -1 // Use -1 or omit the deviceId to select the default device
}
})
const musicBeatScheduler = new MusicBeatScheduler(pos => {
console.log(`peak at ${pos}ms`) // your music effect goes here
})
const musicBeatDetector = new MusicBeatDetector({
plotter: musicGraph.getPlotter(),
scheduler: musicBeatScheduler.getScheduler()
})
ai.start()
ai
.pipe(musicBeatDetector.getAnalyzer())
.on('peak-detected', (pos, bpm) => console.log(`peak-detected at ${pos}ms, detected bpm ${bpm}`))
.on('end', () => {
fs.writeFileSync('graph.svg', musicGraph.getSVG())
console.log('end')
})
.pipe(ws)
.on('open', () => musicBeatScheduler.start()) This produced a 1-beat SVG before exiting with The raw-audio file looks similar: |
I made an issue over here maybe they will have some ideas. |
This works! My only issue now is that the mic input is nowhere near as accurate as piping the audio straight to the analyzer. mic
.pipe(musicBeatDetector.getAnalyzer())
.pipe(fs.createWriteStream("/dev/null"))
.on("open", () => musicBeatScheduler.start()); |
I'm still thinking about this, but don't have too much time to work on it, lately. Any progress? Here are some other ways we might get cross-platform audio + beat-onset-detection: |
Not much progress after the last "breakthrough" haha. I've since been playing with Phillip's Hue lights |
I would like to see something like this done with Electron as it would be more portable (I think?) and rely less on system dependent libraries. |
I think it's very portable with all 3 methods, but electron is probly easiest. I played around with deskgap (a light electron alternative that runs faster) for a while, but couldn't get audio capture to work. I can revisit later with electron. personally, I really don't want a GUI to be required, so C++ or even a python-pipe sounds a bit better. |
fun stuff here :) |
One idea might be to use my new crossaudio to do some analysis. It currently doesn't have native (non-browser) mic-input setup, but should be pretty easy to add, and if nothing else could be used in a browser (like via electron or similar) to send commands to the lights. Pretty sure the spectrum/bargraph stuff I am doing could be used to make lights act in a cool way with sound, even if not used to detect beats. this could sort of directly translate to "make a bunch of colored lights show something cool to music" |
this is a cool use of code to change colors of light, but could be much more efficient if it all ran in node instead of python, and would show off how to use API better:
Need to do some research to resolve
TODO
'sAlso, I should figure out Kyle Dixon's github name (got code from an email they sent.)
The text was updated successfully, but these errors were encountered: