Touching Sound

Touching Sound: technologies for collaborative multimodal musical interaction

Key partners: Phil Barnard (MRC Cognition and Brain Sciences Unit), Ian Cross (West Road Centre for Music and Science), Satinder Gill (West Road Centre for Music and Science), Tommi Himberg (University of Jyvaskyla, Finland), Richard Hoadley (CoDE/DPL/MPA), Helen Odell-Miller (ARU Music and Health Research Group), Gill Westland (Cambridge Body Psychotherapy Centre), Sam Aaron (Cambridge University Computer Laboratory) and Rob Toulson (ARU CoDE).

Issue - benefits of using technological resources in music-therapeutic contexts

In music therapy one objective is often to influence positively the clients' skills in communication and interaction. In the therapeutic interaction, the capacities and musical ideas of the therapist and the client(s) are mediated by musical instruments. With 'traditional' instruments, interaction is encouraged and shared music can lead to improved outcomes such as increased awareness of the other, or the capacity to be more playful. However, in this project, the aim would be to explore the merits of introducing a mechanism requiring shared music making and co-operation by building an instrument that would foreground this interaction, making it the focus of mediation which may also enhance individual capacities. Examples of previous work include 'Soundbeam' (http://www.soundbeam.co.uk/ ), have tended to focus on the notion that movement should be mapped to standard technologies such as MIDI, driving synthesisers and other preexisting units in fairly traditional musical outputs. The results are generally used to enhance individual capacities.

Technology can be used to mediate musical interaction by providing mechanisms that shape collaborative musical action

Non-standard interfaces and computer music systems can be developed that require coordinated behaviour on the part of two or more individuals to manipulate musical sounds, gestures and structures. This will allow for unique interactions to develop between therapist and client, and potentially between clients themselves. Such interfaces can also help to overcome pre-existing and perhaps negative responses to understood interfaces, such as keyboards, etc., through considered and imaginative sculptural design that is sensitive to appearance, texture and the physical metaphor.

Testing of interactive and musical affordances of technology

Experiments will be undertaken to clarify the qualitative implications for the dynamics and experience of the interactive relationship between participants when using technology that mediates their connection. In particular, it is essential to test and define the relations between joint action and sounds/musical structures that are afforded by the interface prior to any therapeutic use.

Applying technology to clinical population

Depending on the results of the testing of the technology, selection of appropriate clinical population who might be expected to benefit from its application might be those with marked communication difficulties, but also those populations for whom music may be a specific intervention. For example, there is little literature presently showing evidence that music therapy is helpful for people with attention deficit disorder (ADHD) specifically, and this population draw upon medical and psychological services and cause, particular at teenage stages, many social and forensic problems. A Cohrane Review is planned in this area and any research such as this would contribute evidence to 'what works'. The older population with dementia is also an area for consideration, where further work needs to be done. Owing to cognitive impairment, music can help shared interaction, but further research is needed in this area.

The Touching Sound GISMO

A Draft GISMO States Diagram

A Draft Extended GISMO States Diagram

A Sample of Collected GISMO Data

June 2011 workshop