For Mr. Swirly's live performances, he has a custom patch named The Commander which is implemented as a Max For Live plugin primarily written in Javascript.

The Commander allows him to control and select from his large library of sounds and effects with a minimal number of gestures and without seeing the computer's display, using a footpedal, the electronic wind instrument, or any other standard or alternate controller.

A paper discussing part of The Commander can be found here. Most of the code is a "trade secret" but Tom has open-sourced some of the more useful Javascript classes and function, and you can find them here.