Long-Haul Composition and Digital Crutches: How I used the virtual performers of the digital age to overcome the impact of post-COVID syndrome (long COVID)

A short long-haul history

To those of you who don’t know me (basically everyone reading this, I imagine), I am a composer currently undertaking a PhD at Royal Birmingham Conservatoire.

As I write this, I’m marking exactly one year since the appearance of my first COVID-19 symptom. Lockdown had already led to the cancellation of the performance of my piece des colombes et des crocodiles… at West Road Concert Hall, Cambridge, and I can’t say I was looking forward to months of isolating restrictions. I had, however, thought that that would be the worst of it, and that I’d have a productive lockdown, finish most of my PhD (which focuses on the role of tools and cognitive extension/embodiment in musical composition), do some press-ups, and learn to bake bread or something.

Instead, I woke up on the 30th of March 2020 struggling to breathe for the first time in my life. It wasn’t like being out of breath from working out – perhaps more like being winded. To this day it remains hard to describe. I was gasping for breath from the effort of sitting in a chair. A total weakness flooded my body, and after a terribly exciting visit by some paramedics in makeshift PPE, I settled in, as it turns out, for the long haul.

For me, acute shortness of breath gave way to considerable muscle and joint pain, exercise intolerance, chest pain, weight loss (which I can’t put back on thanks to the exercise intolerance!), fatigue, brain fog, and an inability to focus. I used to row a lot, and I found myself using the mental technique of dividing an intense workout into manageable chunks: ‘just 100 more meters, just 50 more meters’, for daily tasks like having a shower, making a cup of tea, or struggling down the footpath that runs alongside my road.

No one knew much about COVID-19, back then, and I’d hoped that I could ‘power through it’ in the same way that I would power through any other difficult thing. It doesn’t quite work that way, as it happens, and I quite possibly made things worse by pushing myself to keep doing things. So, I interrupted my PhD, and did nothing except try to get better, with varying success. A year on and walking is still a struggle, I sometimes need painkillers to manage the muscle and joint pain, and I occasionally need to breathe through my evening routine like it’s the end of an intense workout.

However, this article is not about how irritating post-COVID-19 syndrome is, or how frustrating the (thankfully rapidly easing) lockdown restrictions were! It is about overcoming them.

In November 2020, I figured out a way to actually do some musical work, by using virtual instruments designed for media music to record my pieces which had had cancelled performances. I’d had some experience working with virtual instruments in the past, but in the following 5 years the range and quality of these instruments seems to have extended to the point where they could have applications for contemporary classical music.

So, I spent a week planning the digital recording process of one of my pieces to the finest detail. If a task would take more than 5 minutes, it was broken down even further, or put into its own category for when I was having good days. I had this plan in a word document, opened my software, and got to work, highlighting completed tasks in yellow. At the end of the day, I would edit my plan based on how the work had gone. To those of you who are wondering, yes, this is also how I am writing this article.

Some days I managed nothing, some days I managed two or three 5-minute tasks, and on some good days I managed 2 or 3 hours, spread over the course of the day. Having instant audio feedback from the virtual instruments was important for the work itself, but it was also important motivation to keep going, as I had a constantly improving recording to listen to at any point.

By December 2020, I had a finished recording of my Emergence//Overture, written in 2019. By January 2021, I had a high-quality recording of des colombes et des crocodiles…, the performance and recording of which had been cancelled by lockdown restrictions. You can listen to both pieces here. I don’t want to understate the sense of achievement, nor how important it was to be able to share my pieces with others, nor how much this improved my mental state after more than 9 months of illness.

At the end of January, I returned to my PhD part-time (but with no expectation of 20 hours a week, for which I am very grateful to RBC), and with all I had learned about working musically while managing my condition, I decided to try to compose again. Considering the value of the instant audio feedback when recording my existing pieces, I realised that virtual performers could perhaps be a means to continue to compose despite the limitations imposed by my illness.

Leaning on digital crutches.

If you’ll indulge me, I’d like to turn briefly to a thought experiment posed by the philosophers Andy Clark and David Chalmers in 1998. In this thought experiment, Otto, who suffers from Alzheimer’s, and Inga, who does not, are both trying to get to the Museum of Modern Art. Inga thinks for a moment, remembers the address, and heads off. Otto instead relies on a notebook, which he briefly consults. After finding the address, he similarly heads off to the museum.

The contention of Clark and Chalmers was that we should not consider Inga’s internal memory and internal accessing of that memory as inherently different from Otto’s ‘external memory’ and external accessing of it. They contend that cognitive processes can, and often do, make use of objects outside of the brain, and that these objects should therefore be considered as a part of those cognitive processes – in the same way that neurons firing in the brain are widely considered to be part of those cognitive processes.

The metaphor of Otto was certainly not meant to argue that the use of external objects only counts as cognitive if it makes up for some neurological condition. Imagine moving scrabble tiles around to find a better word, rather than trying to do it in your head. Or, for that matter, think of the numerous spatial metaphors in the way we write and talk about music. An extended view of musical cognition might see these as more than metaphors. In the context of composing this new piece, though, I am perhaps more in the realm of Otto; looking for ways to ‘offload’ those compositional processes which now require more time, focus or energy than I have available.

I should of course note the limitations of virtual performers compared with real performers. When writing for a real performer, you’re communicating to a real person, and you can simply ask them, through notation or text, to do things like gradually move their bow from the neck to the bridge or to play in a certain manner. You just need to understand the limitations of performer and instrument. Additionally, real performers will add their own nuances and expression to the written part, no matter how detailed, and will bring their own ideas to the piece in rehearsal.

In making a digital recording, it is necessary to mould every note yourself. And when working with virtual performers, if the sound or technique you want isn’t part of the sample library, you can’t use it. The two recorded pieces were initially written for real instruments, and I had to “fake” some techniques, or re-write them, for the digital recordings. Where I had written quarter tone lip bends in the woodwinds, for example, I had to record the note twice, with the second note tuned down a quarter tone artificially, and then automate an equal decrease/increase in volume from the note in standard tuning to the note which was tuned down. This crossfading between notes tricks the ear into thinking that the note is smoothly decreasing in pitch by a quarter tone.

I feel that the fact that I’m writing this new piece using virtual performers as the primary tool of composition, rather than just of recording, will somewhat mitigate these issues. On the other hand, I know there will be moments where I am unable to use a technique or idea that would have taken me two seconds to write in a score destined for real performers. However, as anything I write in the score will come from the recording made with the virtual performers, the recording and the score will be as faithful a match as possible. I am excited to see the role the digital recording can play as a tool for rehearsing the piece, and to hear the differences between the recording and the first “real” performance from the score.

In a considerable silver lining, I feel that this conscious and documented offloading of parts of my compositional process to virtual performers can perhaps provide a compelling background to the more contentious claims of my PhD research, like the role of imagined performers in creating a part and the development of compositional competencies through the use of tools. In response to these attempts to compose through illness and lockdown, the scope of the thesis has narrowed to focus on imagination, memory, and the virtual; and is much the better for it. If I’m honest I’d rather be able to go for a hike again, but there’s no doubt that my thesis has been improved by my experiences of the past year.

So, by figuring out how to support so much of the compositional process on digital crutches, I had figured out how to be able to compose again despite (and to spite) my condition. However, I knew I would also need to make use of the organisational techniques that had allowed me to record Emergence//Overture and des colombes et des crocodiles…, and for this I turned to Kandinsky.

Symmetry II

I have had a pre-occupation with the work of Wassily Kandinsky since the first time I saw his ‘Swinging’ in the Tate Modern, when I was 19 or so. It was part of the most well curated room I’ve ever seen, which was sadly completely re-arranged when the gallery underwent its Tate re-Modernization in 2018. As I developed as a composer, I often wondered if I could take the symmetry and balance that so drew me to Kandinsky’s work and deal with it musically.

test1

Wassily Kandinsky Swinging (1925)
Photo © Tate
Image released under Creative Commons
CC-BY-NC-ND (3.0 Unported)

I first attempted to deal with “symmetry a la Kandinsky” in a piano piece written for the BFE/RMA conference in January 2020. My composition supervisors, who can never normally agree on anything, both told me it seemed like it should be the start of a series. So, on returning to my PhD, and doing anything I could to follow the path of least resistance, I retitled the piano piece ‘symmetry I’, and decided I would write a ‘II’ and a ‘III’ to finish my portfolio.

For this piece, symmetry II, I decided to create 23 ‘combinations’ of potential sounds from the virtual performers, rhythmic density, and harmonic material. The idea being that these combinations would relate to the multitude of different objects in Kandinsky’s work which somehow feel perfectly balanced – to the point where if you removed or changed one object, the painting would no longer “work”. My thinking was that by dividing the piece into 23 sections, I had already begun the micromanagement and (temporal) division of labour which had allowed me to record the previous two pieces.

When it came to the harmonic material, I tried to create systems which led to “rote” compositional work, like the exercises one might do as part of a composition class and which could be easily broken up into smaller tasks, rather than constant creative decision making. All the material was generated by rotating the intervals of a 5-note chord, always starting on the same note (see Figure 1). I considered each of the 10 resulting chords a ‘cell’ and generated a 12-note row from the order the notes appear.

Figure 1 – harmonic material. Excuse my handwriting.

Once I had this material, I came up with 4 ways of using this material in the piece:

A:

Cell harmonised by consecutive cells.

B:

Solo/duet/trio moving through cells sequentially.

C:

Melodic development of the 12-note row; harmonised by a single cell.

D:

Statement of the 12-note row in some form, with the initial note harmonised by a cell, and then using each successive note as an axis around which to invert the initial harmony. (This technique was inspired by the way that in Kandinsky’s work you can see how the rotation of one object would necessitate the rotation or change of other objects in order to maintain the painting’s balance.)

These harmonic systems would ease the compositional process, as once I made a harmonic decision the resulting work could then easily be divided into manageable chunks.

So, with the virtual instruments chosen and harmonic material generated, I planned a writing/recording process as follows:

  • Invent 23 ‘combinations’ (refinement 0, see Figure 2)
  • Demo those combinations using software instruments, focusing on timbre. (refinement 1)
  • Compose these demos into drafts of sections, including appropriate harmony, using the limitations and potentials of virtual performers as a compositional tool (refinement 2)
  • Combine these drafts of sections into a draft piece, including quotes from other sections. (refinement 3)
  • Create a score based on this draft piece (refinement 4)
  • Record the piece based on the score, as with Emergence//Overture and des colombes…(refinement 5)
  • Make any changes to the score based on the recording (refinement 6)

Figure 2. ‘Combinations’. Some of
these were edited after creating
“demos” with the virtual performers
(refinement 1)

For each stage of refinement, I am making a list of tasks, in the same way as when recording my previous works. There is a substantial difference between composing and recording, though, in that some creative compositional tasks just can’t be subdivided – they are inherently big tasks, or at the very least a continuum of work with no clear subdivisions. However, by waiting for the really good days to make big creative decisions, and by allowing myself to fall into a feedback loop between the recorded draft and my rough vision for the piece, I was able to overcome this.

As I finish writing this article in mid-April, I have completed the rough draft -which is ‘refinement 3’ as per the plan above – and can discuss my experiences.

A potter at the wheel

I have found that the cognitive support offered by virtual performers was every bit as useful as I had hoped it would be. I found myself offloading – sometimes completely, sometimes in part – the work of audiation, sight-reading, knowledge of instrumental techniques, orchestration, and rehearsal. Some ideas were simpler to realise with virtual performers, and so these would easily flow into the piece. Others were harder, and I had to convince myself that the easier ideas were better – or commit to expending energy including them. I suppose there’s a sense in which you could argue that I offloaded an amount of creativity onto the limitations of the tools I was using to compose.

As well as often being led down the ‘path of least resistance’ by what was easy to quickly render using virtual performers, I also found myself making compositional decisions based exclusively on the sound of the virtual instruments. A particularly nice Bartok pizzicato, for example, became the catalyst for the transition between two sections. This specific kind of influence only really relates to writing for real performers in the case of situations where a composer is working closely with a soloist, and they perhaps find an interesting multiphonic, or have a certain quality to their bowing, or have certain virtuosic abilities, which then influence the composition.

Most crucially, though, I found that having instant, malleable audio feedback dramatically eased the cognitive load of the compositional process. I was able to run a section on a loop, editing the MIDI data and instrumental parameters until they sounded right, like a potter at a wheel, shaping clay. And, in broader terms, the whole process of composing the piece follows a similar schema, of feedback and refinement.

Having a malleable image of each section embedded in the scaffolding of the work-in-progress while testing and experimenting – and seeing where the refinements lead rather than aiming towards a piece already fully formed in my head – makes the process really feel like half discovery, half creation. There were no questions of ‘will it work?’, but rather ‘is it working?’. This requires so much less effort, which in turn allows for more work to be done on limited reserves of energy and focus.

All this relates so well to the topic of my PhD thesis, that I can’t help but feel that if I hadn’t been studying the idea of the extended mind and creative feedback loops at the point of falling ill, I might not have stumbled upon this way of working, which makes explicit and conscious the cognitive extension and feedback which is usually unconscious. Just like Otto and his notebook.

As well as the offloading afforded by the virtual performers and the techniques of dividing the labour through the multiple refinements and sections, I was also able to make use of other digital aids. These included using copy and paste to re-create similar parts or when arranging the piece, being able to quickly re-arrange sections to create the draft (see figure 3), and, of course, the ultimate digital luxury of being able to revert to earlier versions if I messed something up.

Figure 3 – arranging sections to create the rough draft.

The perfectionist in me has always struggled to share works-in-progress, but I am sharing the draft – refinement 3 as per the plan discussed above – below. The eagle-eared among you will of course notice some exposed plasterwork, so to speak, and even some scandalous copy and paste. Creating the score (refinement 4) is where most of the refinement of this draft will happen, before the finishing touches are added while making the final recording and tweaking the score to reflect it (refinements 5-6). Now that I think about it, the rehearsal and performance by a real ensemble, guided by the recording, will be the final (and finally collaborative) stage of refinement.

You can listen to the draft here. When I have finished the score and recording, they will be available on the same page. If you are reading this after June/July 2021, they should be there already (he says, hopefully).

While symmetry II originates from the balance of Kandinsky’s work, and the potentials of the virtual string players with which (or dare I say, with whom?) I am composing the piece; it is evolving into something which perhaps captures the intense effort of creating it, and the emotional impact of my changed existence. I therefore felt it deserved to be called more than just ‘symmetry II’. The subtitle, “dare mighty things”, comes from a message inscribed on the aptly named rover, Perseverance, which landed on Mars in February 2021. If NASA can land a robot on a planet millions of miles away while affected by a pandemic, I thought, I should really be able to keep composing.

“dare mighty things”

A few months into composing again and I am confident that, once I recover, my experiences using these digital aids and working methods will help me to compose larger scale works than I have previously attempted. So perhaps I am emerging from all this as a better composer, and certainly with a better understanding of the role of tools in the compositional process. There is also no doubt that I’ve been forced to become a more organised and more disciplined person, as opposed to someone who finishes projects by relying on 14-hour days fuelled by coffee, exercise, and… well, coffee and exercise. And I’ve certainly gained an appreciation for the simple things in life which are, for now, not so simple.

So, in closing, do I think that “dare mighty things” will be the best piece I’ve ever written? I highly doubt it. Dividing the compositional process into so many small sections to make the work manageable makes the piece seem disjointed at times, and there are many places where I’ve taken the path of least resistance in the decision making. I have not once ‘killed my darlings’ – they took too much of my energy to create! The creation of the final score might go some way to rectifying these issues, but, for better or worse, they are woven into the fabric of the piece.

However, just as I can still go for a walk with the help of a walking stick and these walks, while not four-hour hikes, are still better than nothing; with the help of digital crutches, I can at least make the first steps towards composing again.

Thank you for taking the time to read this; and a huge thank you to all those who have helped me through this most difficult year.

Mike

(16/04/2021)


About the author:

Michael Boyle is a PhD candidate at Royal Birmingham Conservatoire under the supervision of Christopher Dingle, Howard Skempton, and Edwin Roxburgh. He is working towards the completion of a thesis entitled ‘Extending the Orchestra of the Mind’, which explores the cognitive processes of musical composition in the context of 4E (extended, embedded, embodied, enacted) theories of cognition.

Contact: michael.e.boyle@cantab.net, michaelboylecomposer.com.

Leave a Reply

Your email address will not be published.