I am working on a game in which I want to use my cello as an input device. I play cello to generate and control the game world, and the audience uses Xbox controllers to play with/against me.
Now it is easy enough to read microphone input from my cello to my game using FMOD (as I was told [url=http://www.fmod.org/forum/viewtopic.php?f=7&t=13521:2he8g35b]last year[/url:2he8g35b]), but what I really want to know is which notes I play and when notes start and end.
So here’s the question: are there any libraries that can do real-time waveform analysis and tell me what notes are being played? Either in the form of a VST (FMOD can use VSTs, as far as I know), or as a separate library that I can call from C++. So far I have not found something suitable, while I know this should technically be possible: Rocksmith and guitar synths must use the same kind of technology.
Bonus features (not necessary, but would be nice to have):
-volume envelope within the note
-how much vibrato there is
-able to recognise very fast notes
-can recognise two-note chords (cello cannot do more than two notes at a time)
So, does anyone have any suggestions for libraries that can do this, or an approach for this?
Thanks in advance!
- Oogst asked 6 years ago
I hand’t seen that one yet, that is actually a really interesting starting point!
I tried it out with my cello, though, and the sample itself is not good enough yet: it is pretty laggy, adds (very short) notes that are not actually being played, often gets the octave wrong and doesn’t seem to handle faster notes all that well. But there seem to be a lot of possibilities in analysing the spectrum array and in using a different array size. Especially since I know the exact instrument I’m playing and that might have some useful characteristics. So this might get me somewhere.
However, I remain interested in hearing any other solutions, since there still seems room for a lot of improvement over this one.
- Oogst answered 6 years ago
Please login first to submit.