Update, August 2012: ABC Classic FM has a podcast of This Is All I Need, together with an interview with Stephen Adams in which I discuss the ideas behind making the piece.

Bionic Ears and Music

Interior Design: Music for the Bionic Ear is a project started in 2010 by the Bionic Ear Institute and Robin Fox to make new music, specifically designed for cochlear implant recipients.

Although cochlear implants (or, to use the technical term, “bionic ears”) are pretty damn miraculous at restoring hearing, they reconstitute sound in ways that make it extremely difficult for users to properly perceive, let alone enjoy, music. It takes patience and training to learn, or even relearn, how to appreciate the musical attributes of sounds.

Watch a short TV segment about the project, or a brief news report about the concert.

The most recent cochlear implant design consists of 22 electrodes, each tuned to a specific bandwidth, from 180 Hz to about 8000 Hz. The arrangement of these electrodes means that harmonies and timbres can end up sounding very differently to a CI wearer.

Tuning and Sketches

For many years I've been interested in making music with tunings different to those found on standard, Western musical instruments. It seemed a natural choice to write a piece of music tuned to a scale which is better suited to the structure of a cochlear implant than those of conventional instruments. The scale used in this piece has 16 tones, instead of the usual 12 tones in standard western tuning. These tones are made up of pitches based on overtones of a fundamental frequency of 250 Hz, the central frequency of the lowest-tuned electrode. They are tuned to produce exact harmonic relationships between notes; however, because each electrode responds to only a certain bandwidth, certain parts of the scale will be received and interpreted differently, depending on which octave you play in.

Having settled on a scale, I made a number of short, simple pieces using this tuning system.

Study No. 1 was made by filtering white noise into the 22 frequency bands used in the design of a cochlear implant. This was done using a filtered granular synthesis contraption in AudioMulch. The filtered sounds produced were mimicked by a (virtual) piano, retuned to the 16-tone scale. The sounds you can hear in the study are a mix of the white noise, the piano, and either or both sounds reproduced through the cochlear implant simulator devised by Robin Fox.

Study No. 2 examines the various harmonies that can be produced with the scale. Using only one instrument (electric organ), a sequence of chords and single tones are played in a variable rhythm. Certain pitches, with frequencies that straddled a pair of electrodes, were shifted up or down an octave. This sequence was fed back into the same AudioMulch filter used in Study No. 1, which plays back differing amounts of the original and filtered organ.

Study No. 3 was made later on in the project. It is a single element that was intended to be incorporated into the final, completed work.

The Moment of Truth

It’s a strange experience, having to play your music to an audience of one and waiting to find out their response, face to face. Even stranger, when they know nothing about your music; stranger still when you know they’re not hearing what you’re hearing.

In November I got to meet four cochlear implant wearers at the Bionic Ear Institute. They had differing levels of ability in perceiving music, and of experience in hearing and playing music. I played each of them my Study No. 2 and got some feedback on whether or not my tuning experiments would have any positive effect.

The tuning seemed to work surprisingly well. The types of chords, and the processed organ sound I had used, weren’t as cluttered and muddy as I feared they might be. All four reported that they could hear chords and harmonies clearly, and that the sounds were, for the most part, pleasant to hear. (By pleasant, I mean that too much muddled sonic information tends to sound like white noise to implant wearers.)

It seemed almost too good to be true when a couple of listeners responded that they could identify the organ sound, hear distinct chords and harmonies, and moreover enjoy them. Previously, they had not found these types of sounds pleasant. This was a much better reaction than I had hoped. It seems that using a just intonation scale instead of standard equal temperament has a big effect on how implant wearers hear music. This could be a useful path of inquiry to follow, examining whether equal temperament is an obstacle to music perception and which tuning systems are clearest.

All listeners could identify the organ sound, although some also heard other instruments in the mix. This may have been due to the synthesised nature of the sound, and the other electronic treatments I had made.

A Question of Taste

As an artsy-fartsy Modern Composer, one of the challenges of this project was having to come up with something that people might want to listen to. When the publicity for your upcoming gig promises a concert “designed to be enjoyed by both cochlear implant users and audiences with normal hearing,” you’re suddenly struck by a conundrum. How do you know whether or not the cochlear implant wearers are hearing something enjoyable in your music, when most “normal hearing” people don’t like your music anyway?

My music’s already been written up as using “bizarre scales”, and I have, on at least two occasions, been confronted face-to-face with the question What Is This Shit? So when making this piece I asked myself, should I try to write something (shudder) “accessible”, or carry on doggedly clutching my copy of “Who Cares if You Listen?” At least I can console myself that the tunings I’m using, which may sound off to most people, seem to sound pretty normal to implant users.

That was the quandary I’d been facing since visiting the Bionic Ear Institute. While I was on that visit to Melbourne I got to see the (fantastic) launch concert for the CD Artefacts of Australian experimental music: volume 2. One of the composers on the CD, Sarah Hopkins, played her music on whirly tubes. As well as her own works, she performed Amazing Grace. Yeah, it’s simple and obvious, but whirly tubes play only notes in the harmonic series. In other words, it’s not in conventional tuning but a “bizarre scale” similar to the scale I’m using in this particular piece.

The scale is also very similar to one used by Ben Johnston, a composer with over 50 years’ experience of writing music in alternative tunings. His best known piece? A set of microtonal variations on Amazing Grace. This string quartet marked a changed in his style, from the more abstracted idioms of the post-war avant-garde, to using familiar harmony and melody as a foundation on which to build sophisticated elaborations on the physics of sound.

Although my music is still very different, I used those examples as a reminder of how I would like my music to be heard: I don’t want it to be easy, but I want it to be clear.

This Is All I Need

Study No. 1 was very rudimentary, and served as a preliminary map of the type of soundworld I am dealing with. Study No.2 was a demonstration of harmonic combinations that are possible. For the finished piece, I considered the followeing principles:

  • Implant wearers report being able to understand speech very well. I'm using a speaking voice as a sort of key, or guide, to the music.
  • Using lighter instrumental timbres with simpler sounds.
  • Building textures that sound active, without becoming dense.

Adding speech fragments to the music reminded me of the simple, disconnected phrases often heard in language lessons.  I started to envisage this piece as a primer in a new language, establishing a basic vocabulary which could be built upon and expanded in subsequent pieces. The spoken phrases inspired four short musical patterns, which imitate the speech's rhythm and melody. 

At first, these patterns are simply repeated while being accompanied by different harmonies. Later, the melodies begin to change while the rhythms stay more or less the same. The chords used to play harmonies are kept as close as possible to those found in popular music.

The music written for the final piece - now titled This Is All I Need - was composed entirely for electronic instruments. This allowed me to keep the music precisely tuned throughout. Also, by using older, simpler synthesiser sounds, I could stop the instruments from sounding too rich and possibly interfering with the listener’s perception of melody and harmony.

As the venue for the premiere contained a multi-channel speaker system, I prepared a spatialised mix of This Is All I Need to further distinguish between the different strands present in the music.

A Detailed Analysis (please skip)

This Is All I Need is made out of four short, spoken phrases. These phrases were transcribed into melodies, with the rhythm and intonation of my voice approximated by the rhythm and pitch of the melody. A list of chords was compiled, consisting of the most consonant harmonies readily obtainable from the 16-note scale. These chords largely fell upon 7 notes in the scale. Each of the four melodies contains two longer notes, which are used as the root notes for the aforementioned chords. The choice of chord for each root note was determined by chance.

The piece is in two parts of roughly equal length. In the first half, the four melodic units are repeated in turn, occasionally in alternating pairs. When repeated, these units may contain any one of, or combination of, the elements of melody, voice, and chordal accompaniment. Variety to these repetitions is produced by the changing chords, the presence or absence of constituent elements (voice, melody), or the alternation from one unit to the next. In the first half, the melodies also appear in transposed form, with the rhythm unchanged but each pitch raised or lowered by chance. These altered melodies are also played by different instruments, and harmonised only when suitable pitches to form a consonant chord are present.

The second half of the piece has a slower tempo, and plays three of the four harmonised melodies in rotation, in a series of permutations that allows all possible combinations of the three melodies to be heard. This extended melody is then altered by chance-determined changes to pitch, while preserving rhythm. The altered melody is also harmonised by accompanying chords. No longer held by the strict recurrence of certain chordal roots in the original melody, new harmonies are introduced to the piece which could not be heard previously. The piece ends with the melody omitted from the preceding section, repeated as an ostinato over which a final, chance-determined series of chords is played, all on a common root heard repeated in the ostinato.

Interior Design: The Concert

This Is All I Need was premiered at Interior Design: Music for the Bionic Ear, at the George Fairfax Theatre, The Arts Centre, Melbourne on 13 February 2011.

Six pieces by different composers were presented, each taking a different approach to making music that may be particularly suited to the bionic ear. All audience members, regardless of whether or not they used implants, were asked to fill out a short survey included with their programmes. Furthermore, after each piece listeners were asked to grade their reactions to the music, based on a set list of questions.

Besides my own work, the other pieces were:

Rohan Drape, Another In Another Dark. This piece for clarinet, viola, cello and piano had a very late-Morton Feldman feel to it; not so surprising when the composer refers to Feldman’s Palais de Mari in his programme notes. The instruments play subtly shifting textures in extended, suspended harmonies.

Natasha Anderson, Study for the Bionic Ear #1. For the first half, two percussionists iterate a cycling rhythmic pattern on tuned drums and shaker, before a sampled piano and electronic noises intrude. The second half focuses on constant sounds from a vibraphone, rolled and bowed, mixed with sampled cello drones and electronic tones.

James Rushford, Tussilage. This piece for viola, cello and tape (playback and electronic sounds) kept steadfastly to the “difficult” language of the avant-garde, with extended playing techniques and a mixture of pitched sounds and noises of varying complexity. The use of these sounds of these sounds was based upon earlier tests and auditions with implant wearers.

Robin Fox, 3 Studies for the Bionic Ear. Electronic sounds were simultaneously played through the surround speaker system and represented graphically on the screen, either as colour bands of pitch frequencies or as waveforms. The sounds alternated between steady drones of electronic tones that accumulated overtones in different patterns, affecting harmony and timbre, and differing articulations of sharply rhythmic, ascending scales.

Eugene Ughetti, Syncretism A. Three percussionists produce an array of timbral and textural effects, largely with untuned instruments, also using amplification and other electronic treatment, as well as speaking voices in one section.

Afterwards

The survey sheets used by the audience concentrated on questions of aesthetic pleasure to be found, or not, in each piece. The results are now being collated and analysed. Audience members were also invited to attend discussion groups immediately after each concert, to give their thoughts and reactions.

It is hoped that this concert, and the feedback data obtained from the audience, can form a basis on which further research can be made to improve the perception and enjoyment of music amongst cochlear implant wearers.

Ben.Harper, 2011.