Introduction

The intention of this unit was to familiarize myself with the use of MIDI as a control tool as well as the transformation of audio into meaningful MIDI data. The chief idea was to create another tool that would be useful for interactive improvisation. Originally, I had envisioned building a pitch to midi convert (using Miller Puckette's fiddle~) which would take any tone I played on sax or clarinet and split it into a chord. With that done, I could assign my foot controller to make different chord styles (minor, major, diminished etc.). I got bored with that, although I'll probably revisit it in an upcoming unit when I've figured out how to transpose real-time audio on the fly to create a true harmonizer.

What I have come up with for this unit is what I'm tentatively calling a 'note follower'. When I play a note (on clarinet), the computer will play back a series of midi notes in a specific but unpredictable relation. This occurs due to an algorithm based (loosely) upon the particle swarm optimization algorithm provided by http://www.swarmintelligence.org/tutorials.php. The idea is to generate pitches which move toward my clarinet pitch.

Method

The patch itself consists of several parts. Pitch to midi conversion (provided by fiddle but slightly modified by me into ptom.pd), two arrays (pitches and velocities), a calculation algorithm and a reading/output mechanism.


ptom.pd - The pitch to midi abstraction, featuring Miller Puckette's fiddle~ object. moses is a greater than/less than object which I have used as a gate.

The pitch to midi abstraction (ptom.pd) is fairly trivial since Miller Puckette's fiddle~ works quite well. I've added the ability to transpose the output as well as a gating mechanism (moses) to cut-off any frequencies under 30 (my microphone and room can be noisy).


Pitches and Velocities arrays while calculating a large intervallic jump from a low held note to a high one.

The arrays and swarm calculator are the central features of the patch. The pitch array holds 10 midi pitch values (from 0 to 127). The velocities array is less straightforward. The values stored in the velocities array represent the velocity and direction that the corresponding pitch will travel on its next iteration. This is not to be confused with the standard midi velocity which equates to volume.

An example for clarification: assume that element 0 of the pitches array (p[0]) is set to 67 (G) and the corresponding element 0 of the velocities array (v[0]) is set to -3. This means that upon the next iteration through the array, p[0] will descend 3 semitones and be set to 64 (E). Velocity v[0] will then be recalculated so that upon next iteration, p[0] will come closer to whatever note I am currently playing.


swarmcalc.pd - The brains of this patch. swarmcalc approximates the direction and velocity of each succesive pitch so that they will tend toward my pitch.

The calculation is a stripped down and simplified version of the particle swarm algorithm and isn't really an example of swarming, I just couldn't come up with any other name. The premise is that given an attractor (my clarinet pitch), the pitches in the array should move toward it at varying velocities. Calculating the velocity is straightforward:
v[n] = 2 * ( attractor – p[n] ) * random
( attractor – p[n] ) represents the distance between my current pitch and the pitch stored in the array at location n. If velocity were set to this difference then that pitch would be equal to my pitch upon next iteration. However, I would like a more gradual approach which is provided by multiplying a random number between 0 and 1. To add a little more variation, the possibility for overshooting is provided by multiplying the entire expression by two.
The other expression is a simple test to see whether the velocity has slowed below 0.3 (which means that the pitch is too close to my own) in which case it recalculates the velocity within a range of 2.5 semitones up or down.

Finally, pitches are iteratively read from the array and output as midi notes at given intervals (between 1 and 387ms). I have attached the playback speed of these pitches and the length of each note to two midi expression pedals on my foot controller. Output velocity of each note is calculated based on my current loudness. If I am below 50, output velocity will be half that and sound soft, if I am above 50, it will be 1.2 times louder. This has the effect of creating a larger dynamic range as well as sometimes providing accents.


MidiFollow.pd - The full patch.

I should mention that the recordings were made by outputting this data to Ableton Live to get more pleasant midi synthesis.

Audio Examples

I made two recordings with this patch to demonstrate its capabilities as a pitch generator. The first example, MidFollow.mp3 is only the synthesized midi output which has been generated in conjunction with my own clarinet playing. When I recorded this track, I hadn't yet figured out how to get the audio from my microphone to record in conjunction and in sync.

The second recording MidExample.mp3 has both clarinet and generated midi playing together to give you a sense of how the improvisation worked and how my performance on clarinet influenced the generation of midi notes.


Pitches and Velocities arrays after a long, held note on clarinet. This is why the pitches are all very close and the velocities do not vary much (they are hovering around 0).

Pitches and Velocities arrays while listening to clarinet playing many intervals (around one octave) which is why the velocities array is so varied.
Discussion

This patch is one step closer to being something interesting to improvise with. It plays around my pitch, if I am jumping up and down in pitch, it will also be more volatile in range and it's also soft when I'm soft and loud when I'm loud. It draws inspiration from the “Transformation in Improvisation” section of Rowe's chapter on interactive improvisation. Much like Amnon Wolman's “systematically random”, I feel that I have created something that “applies random variations of pitch [but not velocity] within a systematic control structure” (Rowe, 298). Though the general actions of the patch are highly predictable, I can never know the exact pitch that will be next.

In cognitive terms this patch is interesting from the interactions of acoustic clarinet and midi machine. (un?)fortunately, Miller Puckette has already designed the simple to use and effective fiddle~ which does an admirable job of detecting pitches so there is no need to belabour the difficulties in teaching a computer to 'hear' pitches. Of note is the fact that the system is not knowledge based and relies on the interactions between clarinet and pitch array. The patch could be modified, however, to allow for scale quantizations so that following notes will snap to particular pitches deemed to sound 'right'.

There are other additions to the patch that could help this patch evolve into an effective improvisational tool or composition. First, machine listening for beat, rhythm and timing information could be very useful. To couple this with the same algorithm used for pitch could produce rather interesting and expressive performance (Rowe, 264). Next, I would like to use a sampler and pitch shifter as the 'synthesizer' for the midi notes. Instead of a saxophone and xylophone duo, you would hear two saxophones in a constant game of tag. Finally quantization to particular scales, pitch sets or rhythm values could mold the system toward more traditional forms of music. I envision this particular patch as an improvisational tool that could work effectively as a portion of a larger system for improvisation.