Hey Rahul,
If you haven’t checked it out yet, you might be interested in our blog post breaking down how Glenn Kotche used Sensory Percussion to control lights/video in his PASIC performance last year. It gets into the details, with screenshots of each SP set, diagrams of the physical setup, and clips of the performance.
Whether you’re controlling lights, video, or some other physical device, it’s almost always going to controlled via MIDI sent from SP (though there are a few programs that can use audio). It’s very similar to controlling synths inside of Logic; you’d create different combinations of MIDI note/CC generators inside of your set and send them to another program via the IAC Bus.
For controlling video, you actually don’t need any additional gear (unless you want to connect to a projector or external monitor). You just need software that speaks MIDI and hosts video. For Glenn’s PASIC set, we used Resolume Arena 7
In the screenshot below (taken from the Glenn Kotche PASIC blog post), you can see that there’s a sequencer set to “random” with 9 different MIDI note generators inside of it, each set to a different note value. These notes were mapped to the first row of 9 video clips inside of Resolume. Then this was repeated two more times with two additional sequencers that hold different MIDI note values and have different input filters, which were then mapped to the two other rows of video clips in Resolume. With this setup, each time he hit the kick, snare, or any tom, it would launch a randomly-selected clip from the corresponding row, with up to 3 clips layered over each other at a time.
Compared to Max MSP, Resolume is the much more plug-and-play, user friendly option. You basically add different source video clips to your composition, then you can drag-and-drop preset video effects onto them and MIDI-map different parameters to control via SP. (envelope → scale is a classic). Max, on the other hand, is its own visual programming language, so it’s really a blank canvas where you can dive in and create generative visuals from scratch and go way beyond Resolume’s capabilities. But it has a much steeper learning curve in my opinion.
Controlling lighting fixtures from the drums, like Ian does in “Spiritual Leader,” requires the use of additional software and hardware, namely the DMX-compatible lighting fixtures themselves and also a DMX interface, like the one mentioned in the thread you linked to. For Glenn’s PASIC performance, we used Lightkey as the software component (you can read about that in the blog post). Once you have all the gear connected, the MIDI-mapping process is pretty similar; you create MIDI note/CC generators in your SP set that send to the IAC Bus. Then, inside of Lightkey, you can choose which parameters of which fixtures you’d like to control with those CCs/note values.
If you go back in the blog, we also have some articles specifically on controlling video and lights with SP that walk you through the process. They both show SP v1, but the process is very similar for v2.
Max and Resolume are the two programs I have experience with, but I know other SP users use Ebo Suite for Ableton Live or even the EYESY video synth to control visuals via SP. Max and Resolume both have free demo versions, so I’d maybe start with one of those and mess around with their MIDI-mapping features to get some basic control going from the drums. It’s a super fun application of SP!
