Sensory Percussion Tools for Max/MSP

Because of how it’s built, the pickup itself functions as a regular microphone. So what the SP software/plugin gets is audio, and then it does its magic on that.

So what I do instead is take the audio directly into Max and do my own stuff with it.

The onset detection was the first thing I worked on, and it took me a while to fine tune the settings to get it somewhere where it tracked as well as the SP software. I think I’ve gotten there, and because it’s bypassing the machine learning stuff, if you want really low latency, you can have it. If you want audio descriptors or machine learning classification, you can have that too, it just takes a bit longer as you have to wait for a useful analysis window to pass before it can analyze and match stuff.

This is more of a little perk thing since it’s easy enough to do. But because the SP pickup is a microphone, it can be processed like anything else. The “audio” that comes from it is a bit shit sounding because it’s really a magnetic sensor, which gives you a fast and clear transient, but doesn’t have a full bodied sound, and is quite hissy.

What I’ve done (similar to this video where I did a similar thing for a hihat contact mic) is create a recording using the SP pickup and a nice mic (Earthworks DM20) and then feed the audio from those into a set of tools for impulse response processing (another tech project I was involved with a few years ago). What it does is create an “inverse eq” of the sound of the SP pickup (more or less) and applies that in a way tries to make the SP pickup sound like the DM20. You end up getting a custom EQ curve, that in this case looks like this:

(green is the raw measurement, black is the smoothed version, and red is the inversion)

The final impulse response looks like this:

(black is the final one, red is a filter to cut some lows, and green is the final step from the previous process)

This isn’t something you’d have to mess with to use it, but since you asked what’s going on, I wanted to explain that bit.

Here’s a video of the same kind of process, along with audio, but for a hihat contact mic with a 3d-printed mount I made:

Actually, not sure if I can attach audio, but here is the impulse response for correcting an SP pickup on a snare drum:
correctionSensoryPercussion.wav.zip (15.5 KB)

I’m going to try to make it use friendly, but sort of Max-facing. If you just want a super plug-and-play option, the native SP software/plugin are there and do what they do well. The bits I’ll build should be able to be put together and used without having to get under the hood for everything, and there will be some more standalone-ish options (like a training/matching thing, and the corpus sampler, etc…), but I’ll try to make it in a way that you can go as deep as you want to go with it.

3 Likes

Here’s some performance footage of the corpus-based sampler in action.

So this is 3094 samples that are all pre-analyzed and then being played by by audio analysis coming from the SP pickup and a DPA 4060. So rather than individually mapping samples to zones or regions (or things like center-to-edge), the loudness, brightness (spectral centroid) and noisiness (spectral flatness) of each attack is used to find the nearest match from the sample database.

Not only is this super expressive, but it’s much easier to setup since you don’t have to decide on what samples you want, you don’t have to setup splits, round robins etc…

3 Likes

Wow Rodrigo, this is incredible.

I’m so into it. How soon do you imagine some of these tools being available? The corpus based sampler is of particular interest to me.

Thanks!

Depending on people’s geek level, I can put up greener versions of things to get people going while I still sort things out.

Stuff like the onset detection and correction are done and all that. It’s the corpus-based sampler, and general machine learning/matching stuff I’m still tweaking and playing with.

For example, with that I recently worked out some spectral compensation, where the way you play not only impacts what sample gets triggered but it also applies the spectral envelope (basically the EQ) of what you played to the sample. Imagine it as a per-sample “matching EQ”. It sounds really fantastic as you get another layer of nuance to the whole thing.

This isn’t a musical demo by any stretch, but so you can see/hear what I mean:

So there’s a couple more things like this that I’m messing around with to figure out what kind of features to put into it.

I do have another corpus-based sampler, though it’s more for mosaicking rather than matching samples one-to-one. (here’s a talk on it if you’re interested, and it can be downloaded (for free) here).

1 Like

Wow, this other corpus-based sampler is so impressive and is actually something I’ve been searching for for over a year now. It pretty much contains every function I could’ve hoped for, especially the pitch quantization (loud/pitch knobs). Reminds me a little of Tatsuya Takahashi’s Granular Convolver, which is a similar device built on a raspberry pi and put into a hardware enclosure. Your device seems a lot more thorough though.

Looking forward to playing around with it!

2 Likes

That is spectacular, Rodrigo! Thank you so much for getting back to me in such details. I am here with a big grin on my face. Love what you are doing, man.

I missed the email with notification on your reply, so sorry for the delay. But if you want someone to test some of this on, please write me!:blush:

1 Like

This is great! I just found this topic because I want to integrate SP into Max/MSP but I didn’t know how. I have some questions, it would me great if you can guide me a little bit.

  • Why are you using SP if you’re using it as a microphone input?
  • How it SP communicating with Max/MSP?
  • What is that LED panel that you have around your snare?

I saw your work (Kaizo Snare, Transducer Snare) and I must say it’s pretty inspiring. Thank you very much for showing us your development :slight_smile:

Hey, awesome, glad I’m not the only one out there trying to do this.

This will make more sense with the answer to the second question, but I’m only using the SP sensor in these videos. I don’t use the software at all. After some initial testing I found that I was getting a bit of latency when sending MIDI from the SP software to Max, and on top of that the controllers (Speed, Velocity, Timbre) are only sent as low resolution (0-127) MIDI values.

On top of that, I wanted to integrate SP with my own samples/samplers and didn’t want to make everything again inside their sampler.

The way the sensor connects to the SP software is as “audio” anyways, so what I’m “hearing” in Max is the same thing that SP software is “hearing”. It’s just a matter of figuring out what to do with that signal.

It isn’t at all in these videos. I haven’t opened up the SP software in a long time…

I did make a simple patch a while back (which I can share if you want) that takes all the MIDI messages from the SP Software and routes it, along with a simple visualizer, for use in Max.

That’s a Novation Dicer. It’s actually a DJ controller meant to side on a turntable. It turns out the radius of a turntable is close enough to a snare that it fits comfortably kind of “inside” the rim. I have a 3d-printed mount that holds it in place and attaches it to the snare.

I’ve tried a few other things, ROLI Blocks, etc…, but most things are either too big or clunky to fit nicely on the snare. I recently backed a thing on Kickstarter that I hope will replace the Dicer. Fingers crossed.

1 Like

I was meaning the sensor! Why do you prefer to use the sensor if you’re not using the software? Would it be better a microphone or some set of microphones?

That’s why I want to integrate the sensor to Max actually. With Ableton I was getting some latency too.

I would love to check out that patch if you’re willing to upload it.

That controller looks amazing, the Novation Dicer looks very handy too. I will get one of those whenever I can.

Thank you very much, Rodrigo. Looking forward to see that patch and your answers. I’m Andrés by the way, I’m from Chile :slightly_smiling_face:

1 Like

Believe me, I’ve tried. The main thing is you get a really clean transient from the SP Sensor. it’s not actually a microphone, but a magnetic “hall effect sensor”, which is what also helps it avoid cross talk from other drums. With the onset detection algorithm I’ve been using, I can get really instant response, I just can’t get the very tight press rolls, mainly because in the acoustic world, the sound of the drum is still decaying, whereas with the magnetic sensor, each attack gives a clean spike to get a signal from.

As a point of reference, the top channel is a DPA 4060, and the bottom channel is the SP sensor:

Indeed. When chasing those last few milliseconds of latency, every piece of the puzzle counts. Their overall approach is cool if you like their sampler and want to work with it, but that’s not for me.

Here you go. There’s a comment field showing what CC messages I assigned all the controller stuff to.

SP Basic Setup.maxpat.zip (7.4 KB)

1 Like

Love this!

Thanks for what you’re doing!

Thanks! Will definitely check it out this week.

Hi Rodrigo,

don’t want to bother but I’m not into coding at all and I’m messing around with the combine 1.0 patch.
how do I create a json file from wav files in order to load custom samples in the patch?

Thanks in advance!
Giovanni

Ah cool. Glad you’re digging it.

This part, for now, has to run in Max itself. But you can use this:
Create Corpus.zip (133.0 KB)

1 Like

Super!

Thanks a lot, will try working with it

it seems something’s missing, it doesn’t go forward with the analysis.

Ah right. It’s not gonna work on Windows (for now). I don’t think the main Max for Live device will work properly either. I may pass sound, but the underlying code isn’t fully compiled for Windows. This will change by the time it’s fully released, but that isn’t yet.

1 Like

I will wait for the official release then!
Thanks for helping though. :slight_smile:

Only moderately related, in terms of chasing latency, have you or anyone you know thought about analog / hardware-only options for processing the audio from the sensor?

After reading your post, I’m thinking about running thru (1) a carefully leveled preamp, (2) a compressor to kill that huge initial transient you show, and (3) some guitar pedals to make weird effects. If it’s interesting, I’ll report.

You’ll still need some phantom power, but your preamp can cover that. The pure onset detection happens really quickly (in Max), with the only latency being your vector I/O settings.

But could definitely be interesting to play with.