This article addresses control signals at audio rate in the context of a working installation, Selector. Selector is an audio-visual installation that syncs audio changes (e.g. hard pan controls, audio triggers) with animations. The work uses OSC messages coming out of Kyma rapidly (i.e. 480 bpm), which mark changes in the audio signal. When OSC messages are received by the visualization software, these messages trigger changes in the animation. Because the Paca(rana) is unable to remain with the installation (an outdoor installation with inclement weather), and because tight synchronizations are required between audio and video (without animating by hand!), Selector challenged how I could incorporate live Kyma control. This article discusses how to pseudo-harness the live power of Kyma without the Paca(rana), by storing control signals in sync with exported audio files.

Exporting Control Signals With Audio Files

While there are many reasons to use CapytalkToSound, this article specifically discusses how this Sound can help synchronize control signals with audio recordings (e.g. for installations or collaborations where a Paca(rana) may be unavailable). To export our control signal in parallel with audio, we can use SelectableAudioOutput after CapytalkToSound. By changing SelectableAudioOutput to any channel between 3-8 (not the stereo file), any subsequent recording made will separate the audio sound file from the audio control signal file.

Figure 1. Recording SoundToGlobalController at audio rate in parallel with sound file.

Figure 1 depicts a working example. The control signal (!Pan) is pictured in the Pan parameter field, which controls the panning of an audio signal (0 is left, 1 is right). The !Pan EventValue is generated using a SoundToGlobalController (pan) that has the Capytalk expression:

((((1 bpm: !BPM) nextRandom) abs) rounded) smooth: !SmoothTime ms

The Capytalk expression evaluates to a 0 or 1 generated on the bpm pulse. To help soften clicks of hard panning, a fade between 0 and 1 is set by the user using !SmoothTime. The SoundToGlobalController Sound is pasted inside CapytalkToSound which does sample-rate linear interpolation on its output. Recording our signal (Action > Record To Disk) will prompt us to save three channels of audio. The sound recording is stored on channels 1 and 2 (left and right, respectively). The control signal (CapytalkToSound) in Figure 1 is being routed to channel 3. Figure 2 below depicts a waveform of a recording made with the previous example. The three waveforms include the separate channel of the control signal, and show how this control signal is synchronized with our audio file at sample rate!

Figure 2. Waveform of stereo audio file plus a third channel of a control signal.

The bottom waveform in Figure 2 is the control signal. The signal represents the EventValue !Pan, where a 0 pans audio to the left, and a 1 pans audio to the right. The top and middle waveforms (left and right audio channels, respectively) align perfectly with the control signal. The below sound file plays Figure 2, randomized panning at control rate.

One limit to this approach, however, is the number of audio channels available when recording. The number of control signal channels is limited by the number of available audio channels remaining. For example, if one records a stereo audio file, then only six audio channels remain for storing control signals.

Using Control Signals With Audio Files

In the installation Selector, the standalone version reads three different control signals at audio rate alongside the stereo sound file. Each control signal controls visuals inside a Processing sketch (processing.org); that is, each control signal triggers pulses of projected light.

An example of one control signal in action with visuals can be viewed in the video below. The video displays a Processing sketch that blinks every time the !Pan EventValue changes between 0 and 1, where 0 is left pan and 1 equals right pan.

A huge thanks to Rudi Giot who openly shared his Processing/Kyma communication sketch with me. I used his sketch as the initial template to link Kyma and Processing. The sketch enables direct access of !Pan and other EventValues without requiring separate !OSC_faders. For recording control signals yourself, you may download the Sound explained in this article (see Figure 1), by visiting the Community Library.