transforming-visual-to-audio

[transforming-visual-to-audio]

Hybrid approaches

Hybrid approaches focus mainly an all of the methods combined but also explore one important thing which I noticed during my explorations and which draws from Dubuffet perception of ambiance described in the influences. The choice to give recordings of drawings a role in the making of the composition draws also from Pauline Oliveros and her idea of deep listening. Below is her really interesting talk with her which explains deep listening process the most accurately:

(uploaded by Radiance: Meditation Talk Show on 15 January 2016)

Even though the final result of drawing - the visual image is still, the process of making it is temporal just as the process of making music. The sound of pencil/pen/pastel during drawing could sometimes act as an intuitive guide for the next moves of my hand. The whole process of automatic drawing is very meditative and the only thing I am focusing on is the sound of the object that I draw with. I have noticed that the sound created by this object somehow kept me very present in the moment, allowing for the subconsciousness to speak up. This is where I find a common ground between still image and temporal music, space where both image and sound always emerge in a temporal motion.

This is why I decided to incorporate recordings of the drawings themselves in my compositional practice. I use them in two main ways. Firstly I use them as an audio guide for musical reinterpretation - rhythmic or melodic. Secondly, I use the actual audio and transform it further, for example converting it to midi information and assigning to different instruments or rhythms.Similarly, rhythms and melodies obtain through sensational approach can be transferred into different sounds or instruments.

Below is the example of raw recording of the drawing compared with the same recording transferred to midi and transformed to percussions.

Raw recording

Recording transformed to percussions

[transforming-visual-to-audio]

Drawings treated as graphic scores

In this approach, drawing from graphic scores idea described in my influences I attempt to treat drawings as a kind of graphic scores. Guided by the two levels of visual/audio mediation described by Cinta Cristia I focus on encoding and transcribing visual colors, shapes, the location of an element on the drawing to musical features like pitch, panning, frequency. Still, just as with graphic score, there is an allowance for spontaneity and chance. The choice of sounds and effects in this approach is mostly defined by the features mentioned above. Of course, this can change by looking at drawing horizontally and vertically what makes it even more interesting.

Levels of mediation (Cristia, 2012)

2. Material level

It is based on the correspondence or association of pairs of constitutive elements and/or analogous parameters. They can be previously established or spontaneously drawn, theoretically justified or arbitrary. Some pairs of elements usually associated include color and pitch (i.e., the higher the sound the lighter the color), color and timbre (i.e., yellow can be used to represent the sound of a trumpet), or graphical shape and sound configuration, as in Kandinsky’s (1991) correspondence between a point and a single percussive sound.

3. Morphological level

It involves imitating macro shape –the structure of a work–, or micro shape –the material configuration– in a different artistic medium. Certain parallelisms concerning the material level and the parameters usually related to the musical score and the conventions associated with it are tacitly established. Needless to say, Western musical notation is the meeting point of music and visual expression. Thus, the association between the position of a symbol on the staff and its pitch (i.e. the higher the symbol according to the vertical axis, the higher the pitch) tends to be used as fundamental for morphological associations (i.e., an undulating line usually turns into a melody that ascends and descends by steps). Symmetrical or asymmetrical relationships and internal articulations among other aspects are liable to pass from one art to the other through this level.

By looking at the automatic drawing below and thinking of it as a graphic score I can image a very spare and chaotic bright sound which scatters all over the stereo image in an uncontrollable manner. If I read this one from up as the beginning down to the end I can think of it starting slightly more delicate, reaching a very peak in the middle. After that one main sound becomes prominent yet still is very fast and sparse.      

This is how it sounds interpreted by myself and treated as a graphic score:

The other example incorporates color. While drawing with colors I do not look and draw with my eyes closed so I am not guided by any preconceptions. The drawing below, when treated as a graphic score and looked on vertically, seems to have a strong and low foundation, represented in blues in the drawing and in bass and drums in music. This interplays with mid and higher sounds which are more scattered and all over the place, there is a sort of rhythmical movement repeating, both vertically and horizontally which can be applied to both low and higher frequencies. There is also a sense of moving from darker to lighter colors what for me musically implies a sense of raising and escalation. By looking at the drawing I feel that at the beginning two contrasting realms existing together and in the end they split and only one remains. This is what I applied in music by taking samples of a very rich piano I recorded and combining them with strong noisy and percussive sounds which intensify over time.

This is how it sounds interpreted by myself and treated as a graphic score:

[transforming-visual-to-audio]

Sonification Approach

Sonification approach is echoing Oskar Fischinger’s explorations with music ornaments.This is a direct and analytical approach which translates an image to musical language, in my case to midi notes. 

Sonification at its core aims to make something at first glance inaudible, audible. It is fairly new practice as it relies mostly on the use of modern technology. The most common foundation for sonification are streams of data. For instance, in 2001 Florian Dombois developed a virtual 3D environment of the Indonesian volcano Mt. Merapi by correlating audified seismograms with geophysically calculated visualisations. The Cologne-based artist Jens Brand endeavoured to create an audification of the earth's surface via satellite with his "G Player" (2004) and offers corresponding devices for domestic use in a kind of postmodern sales show. "Navegar é Preciso" (2006) by Alberto de Campo and Christian Dayé traces the route of the first circumnavigation of the globe led by Magellan (1519-1522) by sonifying social statistics of the countries along the way. With his piece "Séance Vocibus Avium" (2008), Wolfgang Müller ventures beyond the scope of scientific verification: Müller asked musician friends to recreate the calls of extinct bird species, taking historical descriptions as a base. (Schoon & Dombois, 2009)

I was wondering for a while what can give the most accurate sound representation of the drawing and I realized the best way to do that is to extract a very precise data from the image. In order to do that I incorporated the scientific software “ImageJ” which is used to analyze cells on micro levels. The obtained data I then transfer to MIDI information using the website application available: http://musicalgorithms.org/4.1/app/. This application is an online based software which allows transforming any data to MIDI through algorithmic deductions. Now, of course, one again could argue that the MIDI file obtained as a result is not a true or real sound of the image. Yet, still, I want to underline that this is not the aim of this project. Nevertheless, the possibility of sonification of the drawings opens up new avenues for latter compositional experiments and solutions. 

Sonification process step by step

1. Preparing the picture

For this example, I will use one of the automatic drawings I made.

2. Processing the image with “ImageJ” 

When the image is ready I load it into the software and convert it to 8-bit binary.

After that, the software analyzes each particle of the lines visible. The results are displayed in the columns of numbers.

The next step is to save the obtained results and convert them into “.csv” file which displays data in a comma separated row rather than a column. This can be easily done online and when the “.csv” file is ready I upload it to http://musicalgorithms.org/4.1/app/.

In Music Algorithms you can choose, pitch, duration and scale. While I pursue the idea of having no intention in this project I choose the values and algorithms randomly without thinking about their roles. 

The MIDI file obtained by this particular image and played back with digital piano in Ableton sounds like this:

[transforming-visual-to-audio]

Intuitional Approach

The first approach which I want to describe is perhaps the most abstract and subjective. I am focusing on the idea of sensation expressed through free improvisation. To help understand how and on what level painting/drawing can be translated into music, Cintia Cristiá proposed to divide visual to audio mediation on five main levels: 

‘1. Emotional level 

This is the first level of interchange of holistic nature. Painting a picture “letting oneself go” or trying to evoke the atmosphere of a piece of music while listening to it constitutes two examples. It is one of the most commonly employed levels in pedagogical experiences that aim at developing artistic and perceptual awareness.

 2. Material level 

It is based on the correspondence or association of pairs of constitutive elements and/or analogous parameters. They can be previously established or spontaneously drawn, theoretically justified or arbitrary. Some pairs of elements usually associated include color and pitch (i.e., the higher the sound the lighter the color), color and timbre (i.e., yellow can be used to represent the sound of a trumpet), or graphical shape and sound configuration, as in Kandinsky's (1991) correspondence between a point and a single percussive sound.

 3. Morphological level 

It involves imitating macro shape –the structure of a work–, or micro shape –the material configuration– in a different artistic medium. Certain parallelisms concerning the material level and the parameters usually related to the musical score and the conventions associated with it are tacitly established. Needless to say, Western musical notation is the meeting point of music and visual expression. Thus, the association between the position of a symbol on the staff and its pitch (i.e. the higher the symbol according to the vertical axis, the higher the pitch) tends to be used as fundamental for morphological associations (i.e., an undulating line usually turns into a melody that ascends and descends by steps). Symmetrical or asymmetrical relationships and internal articulations among other aspects are liable to pass from one art to the other through this level. 

4. Textural level 

Related to the previous one, this level specifically refers to the internal or synchronic relationship of the materials between themselves. 

5. Conceptual level 

Finally, this is the case where the migration from one field to the other is achieved by resorting to an idea or a concept.’ (2012)

With the sensational approach I am focusing mainly on mediating the drawing on an emotional level proposed by Cristiá. While looking at the drawing I begin to play the unfamiliar keys (MIDI controller) or real piano or rhythms. I am influenced by the shapes, spatial relations, colors, emotions felt while looking at the drawing. There is no one particular factor that determines the movement of my hands, and I am allowing myself for freedom of the musical result, strengthening the importance of unconscious decision and spontaneity during artistic the process. I am resigning from deliberate control and encourage the unconscious imagery to pop up. Echoing Cardew’s words, I am not thinking anything up, and in my case, the drawing itself is the heart of the experiment : 

‘We are searching for sounds and for the responses that attach to them, rather than thinking them up, preparing them and producing them. The search is conducted in the medium of sound and the musician himself is at the heart of the experiment’ (cited in Sansom, 2001, p.31)

Below are two examples of my improvisations using keys (one piano and one Prophet synth) on two of the different automatic drawings I made. What is really interesting is the fact the drawings were taken within a few days from each other. Both the drawings are quite similar visually and both the resultant improvisation, while still quite different, share some similarities, and sometimes are in tune and sync, when played simultaneously.

Further experiments with sounds obtained include experimenting with timbral qualities and effect that would pursue the sensations of the drawings. For example using reverb to depict a sense of space or delay to depict a sense of repetitiveness. 

[transforming-visual-to-audio]

Main approaches

My main approach towards composition in this project is to make one long piece of music based on foundations which would not exist without the involvement of drawings. I want to challenge myself and my well-known musical habits and aspirations and make a composition which does not have to ‚be’ anything or sound ‚anything’. Therefore, in the first instance, I am resigning from singing and playing guitar which I am familiar with. I will use instruments or non-instruments which I am not familiar with (eg. keys) and therefore I will not be able to produce my remembered or favored chords, melodies, phrases. I will use Ableton to achieve the final composition, in order to be able to combine all of the different approaches together. For the first time in my musical experience, I want to approach compositional practice without any particular intention or aim. By throwing myself into these new, exploratory compositional approaches I want to deprive myself of notions on creating a musical piece that is supposed to sound plausible, listenable or be of a particular mood, genre. I have no idea what the final piece will sound like, and I am positioning myself in a place where I accept any outcome of these inter-medial explorations. The way the musical piece will evolve will be determined by the elements acquired through different approaches derived from drawings. I divided these approaches into four main sections detailed above.

[transforming-visual-to-audio]

Doubts, concerns and criticism

As Thijs Vroegh notices, there is a certain consensus when talking about the particular type of intermediality which relates to music or film inspired by visual art. He proposes to use the term ‘intermedial transposition’ (2009,p.22). As the author observes, this type of ‘intermedial transposition’ is associated with:

‘the fact that there is an ‘original’ (source) that is conventionally distinct from the newly formed media product (target), where both media have different semiotic systems and, hence, that there is an inter-semiotic transposition.’ (Ibid.)

He then proposes that, with regards to music, the ‘intermedial quality is primarily located in the space between the two works, meaning in the process of gestation but not in the end product, (Ibid.).

I believe such claim is true to my project as the drawings determine the musical processes. Yet we should not forget that this type of intermediality also faced doubts, concerns and criticism.

Music is usually thought to be best to represent actions in time due to the temporal nature of its reception. In contrast to the music, most painting/still imagery is seen as its opposition. The question that seems to arise is a questionable possibility of the musical composition to reflect a painting, or vice versa, whether a painting can be a visualized musical composition. (Vroegh, 2009, p. 43) 

According to Daniel Albright’s, Theodore Adorno or Gotthold Ephraim Lessing would openly speak about their disapproval. Adorno compared such practices to “pseudomorphosis” (Rogers, 2013, p. 56). He warned that:

‘[t]he moment one art imitates another, it becomes more distant from it by repudiating the constraint of its own material, and falls into syncretism, in the vague notion of an undialectical continuum of arts in general. (…) The arts converge only where each pursues its immanent principle in a pure way.’ (Adorno cited in Vroegh, 2009, p.43) 

Elsewhere Daniel Albright with ironically stated that:

 ‘(…) every act of transmediation is a contamination; space and time are mortal enemies. It seems that painters would do well to be deaf and illiterate, and musicians blind and aphasic.’ (Ibid.)

While these doubts remain with no doubt important as Roger’s observes on the example of Klimt and Beethoven artistic correlation, often the aim is more like response rather than simple reiteration. (2013, p.56) One of the plausible explanations and justifications of such intermedial exploration is the synthesis of audio and visual on the multi-sensory level, proposed by Michel Henry in his book ‘Seeing the Invisible: On Kandinsky’ (2009). Henry suggests that we react to the world with our different senses yet we still live in a single world where all of the above are connected. Our world is not divided into visual world, aural world but in a stitched-together reality where we experience it simultaneously. Therefore, there is a factor that unifies all of our senses and our experiences, and the answer lays on a metaphysical level. (Henry, 2009, p.112)

[transforming-visual-to-audio]

Visual art and music: 'sounding’ the image

I believe it is true to say that the main idea of my project is more or less the opposite to described below principles of visual music. It is also connected to the idea of graphic score. What I want to underline is that I am not aiming for a literal music transcription of the drawings but rather I want to use them as stimuli for new musical explorations. Echoing the experiments of Paul Klee, where he uses music as a secondary medium not to elucidate but to manifest the process of art, rather than the art itself. (Rogers, 2013, p.59 ) The intermedial practice of ‘sounding’ the image is relatively narrow comparing to, for instance, visual music, what makes it very fascinating to explore. While such practice has been with no doubt raising many concerns there are still interesting examples of musician inspired by the impact of an image on musical composition. A great amount of discussion and examples can be found in Holly Roger’s book “Sounding the Gallery: Video and the Rise of Art-Music”.

Among the most famous examples of visual art influencing music we can find Franz Liszt (1811- 1886) who composed around a dozen of musical works directly influenced by visual art. His work “Lo Sposalizio” (1839) was influenced by Raphael (1483-1520) painting “ Lo Sposalizio”. Joan Backus discussed the relationship between art and music in this piece in this way:

‘Liszt’s musical procedures can allow a subtle interplay between the musical form an external idea, a reciprocal relationship in which each contributes to the creation of significant and expressive musical form. Sposalizio offers a particularly illuminating of the way this concept of musical perspective functions in Liszt’s music. (Backus cited in Arnold 2002, p.84)

Franz Liszt: “Sposalizio” performed by Andreas Loannides (uploaded by aisrmc on 3 January 2016)

Another famous example of music inspired by visual art is Modest Mussorgsky (1839-1881) piano suite Pictures at an Exhibition (1874) influenced by the artistry of Victor Hartmann (1834-1873), a close friend of Mussogorsky. The composer completed his piece within six weeks from attending one of Hartmann’s exhibitions. The composition consisted of ten short pieces, each of which detected a different sketch of watercolor from Harmann’s collection.

           Mussorgsky  “Pictures at an Exhibition” performed by Chicago       Symphony Orchestra (1980) (uploaded by mahlerman77 on 22 May 2013)

Much more analytical and direct approach to sound the image can be found in Oskar Fishinger’s experiments “Sounding Ornaments” (1932). As described in Holly Roger’s book, Fishinger main aim was to encode “graphic sound ornaments” along one edge of his film strip. These “ornaments” were meant to make a sound when played through a machine, allowing the composer to “speak for itself directly in the film projector”. The aim was to sound the abstract images. (Rogers, 2013, p. 71). Fishinger wrote down his observations about the experiment:

‘In reference to the general physical properties of drawn sounds, we can note that the flat and shallow figures produce soft or distant-sounding tones while moderate triangulation give an ordinary volume, and sharply pointed shapes with deep throughs create the loudest volume. Shades of gray can also play a significant role in drawn music-ornaments’ (Fischinger 1932 cited in Rogers 2013, p.71) 

What Rogers notices in this approach and what reminds me of my sonification approach in this project, is the fact that, Fishinger rather than subjectively interpreting a work, physically determined his results through a machine. 

Oskar Fishinger “Sounding Ornaments” (1932) Experiments (uploaded by CVM on 31 December 2014)

[transforming-visual-to-audio]

Visual art and music: visualizing music

 ‘Shades of colour, like those of sound, are of a much finer texture and awake in the soul emotions too fine to be expressed in words’ Wasilly Kandinsky (1914)

Visualizing music

Visual music refers to many art areas and has its beginnings in the 20th-century. The history traced by visual music includes photographs, color organs, installations, light shows, paintings, films and digital media. According to Jeremy Strick, as describes book “Visual Music” (2005), what connects all of these different art areas is the idea of synaesthesia: ‘the unity of senses an, by extension, the arts’. (Strick in Brougher 2005, p.15). The principle of synaesthesia, therefore, is the manifestation of sensory perception of one sense in another. The most popular example is the phenomenon of seeing colors when one hears a certain sound. (Ibid.) What Strick also describes, and what I find the most fascinating is that the importance of synaesthetic experience lays in the ‘interpenetration of normally unrelated experiences and associations’. In this broader sense, such synaesthetic experience as a creative process rather than the effect is one of my main area of explorations for this project.

The phenomenon of synaesthesia between visual art and music had its beginning in the early 20s and was essential to the development of abstract art. The theory of synaesthesia tented to split perception into discrete units. Music, which consists single notes, phrases, harmony, dissonance and compositional structures was an aspiring ‘candidate’ for a foundation of synaesthetic experience.

Paul Klee, for instance, was fascinated by the musical rhythms and explored them visually in his works, such as ‘Ancient Sound’, ‘Abstract on Black’ or ‘Pastoral (Rhythms)’.

                                    Paul Klee “Pastoral Rhythms”

(Image taken from: http://artmight.com/Artists/Klee-Paul/Klee-Pastoral-Rhythms-1927-Moma-NY-231395p.html)

Wasilly Kandinsky, on the other hand, created synaesthetic paintings which were supposed to be ‘heard’ by the viewer, and ‘of which words could not explain’ (Pocock-Williams, 1992, p.29). An example would be his painting “Green Sound” (1924). Kandinsky’s quest for synaesthetic experience was, in the words of T.Phillips ‘how to paint a symphony?’ (cited in Nelson 2015). Kandinsky’s significant statement on the relationship between art and music can be found in ‘Concerning the Spiritual in Art’. This is where he writes:

‘Generally speaking, colour is a power which directly influences the soul. Colour is the keyboard, the eyes are the hammers, the soul is the piano with many strings. The artist is the hand which plays, touching one key or another, to cause vibrations in the soul.’ (Kandinsky, 1912, p.25)

Kandinsky has been hailed as a ‘visual musician’ and in his works, he was striving to prove the analogy between musical rhythms, harmony and lines, shapes, colors. The analogy is also very apparent in a way he often titled his paintings, below is his painting “Composition VII” (1913)

                               Wasilly Kandinsky “Composition VII”

(Image taken from: https://www.wikiart.org/en/wassily-kandinsky/study-for-composition-vii-1913)

Graphic Scores

When talking about the Kandinsky’s and his interest in the analogy between music and painting, it is important to mention the idea of graphic scores.

Graphic scores were a revolution in the traditional music notation system and their main aim is to notate music differently, through the use of visual symbols, unlike to traditional five-lined musical stave. Graphic notation evolved mostly in the 50s and as John Evarts states in his essay “THE NEW MUSICAL NOTATION -A GRAPHIC ART?” (1968) it was a natural evolution, justified by the broadening palette of musical possibilities, tonal qualities, and the emergence of new instruments. (p. 407). Graphic scores are a great example of more or less direct relationship between still visual imagery and temporal music. They were often accompanied by written commentary from a composer himself for his performers and what Evarts suggest is important, they bring a lot of freedom to the performance and also a great amount of chance. As the notation is looser than with traditional notation, Evarts claims one work will always differ from another. Therefore, every time the score is ‘realized’ it creates a new and unique version of the piece. (Ibid.)

What is personally the most fascinating to me is the fact how each graphic score differs from another and there is no such thing as a rule which can be applied to make one. Everything is limited by composer’s vision. What I particularly like about the idea of graphic scores is one’s opening for musical experimentation. Of course, one could argue that idea very idea of graphic scores seems far-fetched and its relation to music is too vague. Nevertheless what I find particularly interesting is, as Evarts notices the fact that, graphic scores open the field of observation and interpretation: one is more open for spontaneity, action, and decomposition. (Ibid.).

                                        Anestis Logothetis “Odyssee”

The composer commentary:  'The composition is built of two elements of motion. One is continuous and forms a "path", which runs in vertical and horizontal segments.... Performers in three groups: one group plays the first element of motion-the "path". Simultaneously, the other two groups perform the fields to the left and to the right of the "path".... The smallest number of players is nine; they may select their instruments according to their own wishes’ (Logothetis cited in Evarts 1968, p. 409).

(Image taken from: https://guardareleggere.files.wordpress.com/)

                                 Robert Moran “Four Visions”

The composer’s commentary: 'The performers may begin at either side of the individual movements and read directly across to the other side.... Each performer has a full score and reads directly from each musical structure.' ( Moran cited in Evarts 1968, p. 409).

(Image taken from: https://llllllll.co/t/experimental-music-notation-resources/149/186)

Contemporary practices  

One of the two very interesting contemporary practices which connect still visual art and music are Charlotte Hug’s “Son-Icons” and 3D sculptures made from sound waves. There are many more examples yet I decided to focus on these two as they reflect my practices for these project the most and explore intuitional and much more analytical approaches.

Charlotte Hug “Son-Icons”

Charlotte Hug’s work which reminds me my approaches and explorations for this project the most. Charlotte Hug’s work also with visual image and music and the main difference in her practice and main is the fact that she draws the visualizations of the music on paper, whereas I attempt to do completely the opposite. Her work was an important inspiration for me and to give a description of it I will cite Charlotte’s own words:

‘1. Touching music

I draw my inner visualisations of the sounds using both hands. Through the visual-corporeal track of the drawing process I touch the music.I often use several graphite pencils simultaneously, following the primal musical impulse – the musical gesture – and transcribe this through drawing. What emerge are polyphonic or sometimes even orchestral drawn structures. The Son-Icons serve again as visual stimuli for improvisations and compositions.

2. Playing with musical-visual ideas

The Son-Icon may be turned on its head, the music may be viewed from underneath – the imagination has no limits.

The musical result will, however, always be sustained and imbued by the inner rigour or the sensuous pull of the formal language of the Son-Icons.

3. Combined and Self-Contained

The Son-Icons exist in and of themselves – a harmonious whole. Nevertheless, they cannot come into existence in this form without their interaction with music. The same goes for the music. The aim is not an unconditional joining of drawing and music, but the recognition of visual and acoustic ways of thinking and qualities, which accordingly generate other ideas and other worlds. Nevertheless, at certain moments they both enter into an intense symbiosis.

4. The open air settings or distinctive

locations imbue the genesis of the music and the Son-Icons. Both are strongly influenced by their respective places of origin. Qualities such as those found underground, in wind regimes, rain, flooding, ice etc, influence the music as well as the Son-Icons. Visual traces of the places are also apparent in the drawings (for example signs of abrasion, water etc). These structures in turn shape the music.

5. The intervals between each medium

offer new spaces for thought and creativity - and hence great potential for renewal.The eye often perceives differently to the ear. Where the eye and the ear incline to different results and questions, then the switching of media is an especially strong catalyst for ideas. Equally, any slight incongruence in the two media may continue in unpredictable and often surprising and harmonious ways.The reciprocal process of composition, improvisation and Son-Icons is not linear but branching or fragmented.’ (Charlottehug.ch, 2018)

                                       Charlotte Hugh “Son-Icons”

      (Image taken from: http://www.charlottehug.ch/e-son-icons.html)

More available at Charlotte Hug’s website: http://www.charlottehug.ch/e-son-icons.html

3D Sound Wave Sculptures

I came across these sculptures quite recently as it is still a fairly new practice. The main idea of 3D sound wave sculptures oscillates around making a physical, tangible object from an intangible sound wave. The sound wave could be anything, a song, a certain sound, anything one’s imagines.

Experimental digital media lab http://www.realitat.com/ founded by Juan Manuel de J. Escalante, has created 3d-printed visualizations called “Microsonic Landscapes”. These include different artist’s including Jewels by Einstürzende Neubauten, Another World by Antony and the Johnsons, Pink Moon by Nick Drake, Third by Portishead, and the composition “Für Alina” by Arvo Pärt.

The very process is really fascinating and resembles the opposition of a sonification. I want to focus on sonification as one approach during my project, therefore, I find it really interesting how obtained data can be turned into a different medium. Realität uses open-source three-dimensional data visualization software and then print obtained 3D object with 3D printer. As one could expect, calmer and more ‘floating’ sounds results in condensed shapes while multi-layered, noisy or distorted sounds result in spikes and steeps.

                                “Microsonic Landscapes” by Realität

(Images taken from: http://www.realitat.com/2013/selected_work.php?lang=&nick=8256&tit=MICROSONIC%20LANDSCAPES)

Below I also found a very interesting TEDxTalk focusing on the idea of showing music instead of telling music. 

(uploaded by TEDx Talks on 23 October 2014)

[transforming-visual-to-audio]

Art and unconscious, historical background: Surrealism and automatism

Drawing from psychiatry, Art Brut and mediumistic creation, French artists try Andre Breton, together with his friends developed artistic automatic techniques. According to David Maclagan, automatism refers to a ‚complex series of behavior carried out in the almost complete absence of consciouses awareness’ (2014, p.81).

The first work to apply automatic techniques in French artist’s practices was Breton’s and d Philippe Soupault's ‚Les Champs magnetiques' (1919). As described by Gibson, Breton, and Soupalt before approaching writing would empty their mind from any conscious stimulation (willfulness) or external stimulation (environmental stimulation). They wanted to create a state as passive as possible in order to give the mind the full space. After that, they would wait for phrases to enter their mind and these would be immediately transferred onto paper (Gibson, 1987, p.56). Later from this point, Breton together with his friends — Soupalt, Cravel, Aragon and Desnos were exploring more creative techniques focusing on unconscious such as automatism, hypnotic trance or dream narration. In ‚Entree des mediums’ () Breton has clearly stated that during automatic writing, described by him as ‚magical dictation’ was the unconscious. ( Breton cited in Gibson 1987, p. 56). As a result of his experiments with automatism, Breton proposed a tentative definition of the movement:

‚What our friends and I mean by surrealism is known, up to a certain point. This word is not of our invention and we might very well have left it to the most vague critical vocabulary, but we use it with a precise meaning: we are agreed it designates a certain psychic automatism, a near equivalent to the dream state, whose limits are today quite difficult to define’ (Breton 1919 cited in Gibson 1987, p. 56)

Equalivent to Surrealist verbal automatism was visual automatism — mainly automatic drawing. Before Masson met Breton in 1924, he completed several automatic drawings which he stated were created without premeditation. (Gibson, 1987, p. 56). His first drawing considered as automatic is ‚The Study for Nudes and Architect’ (1923-24). Masson’s automatic drawings from the beginning of the 20s depict networks of free lines in which fragments of human bodies, animals, objects and geometrical shapes are loosely tied together, giving the sense of what Maclagan calls ‚an all-devouring and violent energy’ (2014, p. 106). Masson described the creative process of making those drawings in these words:

‚Physically, you must make a void in yourself; the automatic drawing taking its source in the unconscious must appear as an unforeseen birth. The first graphic apparitions on the paper are pure gesture, rhythm, incantation, and as a result: pure scribble. […] When the image appears one must stop. (Masson 1961 cited in Maclagan 2014, p. 107)

However problematic is the actual pure connection to the unconscious with automatic drawing, Surrealists practice attempts to getting as close to it as possible. Nevertheless, ad Maclagan proposes, in the instance of automatic drawing, there can be as much as several areas of an unconscious that can come up. (Maclagan, 2014, p. 108):

‚Kinaesthetic unconscious’

This is when a spontaneous drawing is likely to reveal subliminal habits of the drawer’s hand, similarly to the instance of handwriting.

      2. Unconscious ‚symbolism’

Symbolism unconscious in a psychoanalytic sense, with its repertoire of disguised sexual forms: but this yield only a fairly restricted and impersonal range of meanings.

      3. Archetypes and the collective unconscious

Referring to Jung, Maclagan describes this instance as a realm of mythological and transpersonal dimension (Ibid.)

Therefore, my project focuses on automatic drawing practices for two main reasons. Firstly, to deepen and discover the extent and role of the unconscious in a visual creative process and to apply it to amusical process. Secondly, in order for the drawings to create a completely new stimulus for music. Hence, I did not want to have any preconceptions towards the drawings.I wanted to observe and explore them as they are, and for music to be the effect of these explorations.

                             Andre Masson “Automatic Drawing (1924)

(Image from: https://surrealismfall2012.wordpress.com/2012/11/16/andre-masson-and-the-automatic-drawing/)

                             Andre Masson “Automatic Drawing (1924)

(Image from: https://surrealismfall2012.wordpress.com/2012/11/16/andre-masson-and-the-automatic-drawing/)

                  One of my own automatic drawings made for the project

[transforming-visual-to-audio]

Art and unconscious, historical background: Spirit Art

Another very interesting sector of creation where unconsciousness and lack of intention plays a very important role is so-called „spirit art”. According to de Lefayette, spirit art can be divided into few certain characteristics (2017, p.76):

Trance-state:

A large number of artists drew or painted in trance/semi-trance or trance-like state. For example, Polish medium artist Marian Gruzewski painted in a very fast and chaotic series, and he created an avalanche of paintings including imagery visions, events, landscapes and many, many more. Such trance state and alleged lack of control over creation are seen by psychoanalysts as the effect of hallucinations, temporary brain dysfunctions or the takeover by the subconsciousness.

      2. Painting in total darkness:

In many instances, medium painters and spirit artists paint in total darkness, always very chaotically and rapidly, thus evidencing lack of control on the part of the medium (De Lafayette, 2017, p. 92). Heinrich Nüsslein painted almost 2,000 artwork while staying in complete darkness. What seems even more astonishing and extraordinary is the fact the author was allegedly blind. 

                           Heinrich Nüsslein “Untitled (Grey Angel)”

(Image taken from: http://www.tate.org.uk/context-comment/articles/hidden-hand)

      3. Speed

The astonishing speed of the psychic drawings remains one of the most remarkable characteristics of this phenomenon. For instance, Frederica Hauffe (1801-1829) was able to produce complex geometrical shapes and diagrams in less than 10-minute time while John Ballou Newbrough (1828-1891) often drew very fast with two hands at once.

      4. The composition, structure, methods, techniques, and styles of medium artists:

According to De Lefayatte, composition, methods and techniques used by medium painters and psychic artists were unorthodox and alien to traditional norms (2017, p.111):

●  Augustine Lesage usually started his drawing at the very top of the canvas and gradually worked his way down, regardless of the size of the canvas and/or the subject he was drawing.

●  Frederica Hauffe drew complicated geometrical designs and figures, and complex diagrams in all directions.

●  John Bartlett always started his drawing at the left top corner of the canvas.

                            Augustin Lesage  “La danse du pinceau”

(Image taken from: http://cequimeplait.eklablog.com/augustin-lesage-a114276254)

        5. Automatic writing/painting/drawing

This is when a medium painter is an altered state of consciousness and allegedly connects with metaphysical creatures and spirits. The automatically created work is a „transcript” of this communication. The fascinating instance of automatic drawing was profoundly explored by Surrealists and is the one which is crucial to my project. 

                  An automatic drawing by medium Madame Fondrillon

(Image taken from: https://www.christies.com/lotfinder/Lot/madame-fondrillon-dessin-medianimique-5949631-details.aspx)

[transforming-visual-to-audio]

Art and unconscious, historical background: Art Brut

A significant interest of the unconscious and its role started in the first half of the twentieth century. This is when avant-garde movements such as Dada, Surrealism or Art Brut started to emerge and often relied on unconscious as a main guidance and source for the creative process. One of the most groundbreaking and fascinating inspirations for artists at that time was Dr. Hans Prinzhorn’s publication ‚Artistry of the Mentally Ill’ (1922). The publication comprised of almost 5,000 artworks made by schizophrenic patients. The mentally unstable art was associated with free, unconscious impulses, therefore with freedom. Similarly, reflecting Neumann’s concept of ‚archaic primitive symbolism’ artists drew inspirations from different areas such as infantile, primitive, trivial. All of these new artistic explorations depicted what Pomerantz calls ‚unmediated states of consciousness, and represented, then ,the ideal source for a way out of modern, repressive straits.’ (Pomerantz, 2007).

Art Brut

Outsider art (Art brut) has its origins in 1945 when French artist and writer Jean Dubuffet (1901-1985) began his search for art ‘unscathed by artistic culture.’ (Dubuffet, 1988, p. 31-33). Jean Dubuffet was particularly interested in art which diverges from cultural norms and expectations, which ignores canon, traditions or any criteria. While I’m aware that Art brut does not focus exclusively on an unconscious, non-intuitional art, it undeniably reflects a strong individual creative impulse which is free of communicative conventions. (Cardinal, 2009).  This is where an unconscious and non-intuitional approach to art making is important to art brut — states of trance, madness, wildness or intoxication were said to depict what is most creative in a human being — because the most instinctive and direct. Therefore, Jean Dubuffet was looking and showcasing art made mainly by people from social margins like street prophets, prisoners, but also kids and in particular people mentally ill. Art brut gives the impression that artists had no intention whatsoever for their artworks to be visually or aesthetically attractive, which seems to distinguish it from naive art. What seems the most fascinating and important in Art brut is its emphasis on individualism and separation from society. Art brut, similarly to Surrealism looked at a condition of creation where the final is the effect of an uncontrolled, self-taught and self-made process; a spontaneous ‘transcript’ of vision, imagination. Pictures, graphics or any form then is a way to express certain emotional value. Therefore,  the form of creation is subdued by inner stimuli. Art brut seems to be such a ‚transcription’ of an inner self.

As Jackowski notices, the importance of Art brut lays in the ignorance of generally known and accepted means of creation, however, such ignorance is not intentional. (1994, p. 62) According to him, it is crucial to bear in mind that everyone, with no exception, is mediated or influenced by outside world in some way. Hence, also in the instance of art brut, the creative work is a resultant of a level of technical ability, personal, cultural or artistic experiences. Nevertheless, in Art brut the distortion in the relationship human — outer world leads to limited influence of outer world at the final creative work. Hence, originality of Art brut works can be seen in the lack of interest in conventional means of creation and outside norms. (Jackowski, 1994, p. 62).

Among figures coming under the artists of Art brut we can find Adolf Wölfli, Judith Scott, Nek Chand, Felipe Jesus Consalvos and Henry Darger.

As psychiatrists Walter Morgenthaler said in his monograph on Adolf Wölfli, who was a patient in a mental institution: 

‘Wölfli never created in accordance with an ideal, but entirely in response to his instincts. He doesn’t know the laws by which he works, but he obeys them unreservedly...  When he creates, it is in a state at once sublime and oppressed, ruled by a powerful inner tension, by something concrete, serious, and measured, yet fully personal, violent, and turned towards the absolute.’ (Morgenthaler 1921 cited in Maclagan 2014, p.82)

                                 Adolf Wölfli “Die Dampfer-Schraube“ 

       (Image taken from: http://www.marold.cz/blog/obrdilo-adolfa-wolfliho)

                                           Judith Scott “Untitled”

 (Image taken from: http://www.laboiteverte.fr/lart-de-la-pelote-de-judith-scott)

                        Felipe Jesus Consalvos “National Lullaby”

(Image taken from: http://www.fleisher-ollmangallery.com/artists.php?id=10&page=1&img=1)

An interesting area of the Art brut and the one which seems valuable to my research is the „musique brut” as described in David Troop’s book „Into the Maelstrom: Music, Improvisation and the Dream of Freedom: Before 1970”. Aforementioned pioneer of art brut — Jean Dubuffet was also a musician. His interest in characteristics of art brute found its reflection in a way he approached music. What is important, he differentiated music we make and music we listen too. He underlined the fact that the first one was a composite of ‚inner moods and motivations, mixed with the everyday ambient sounds that are involuntary constant of living.’ ( Troop, 2016, p. 217).

Dubuffet’s musical preferences focused on music „not structured according to a particular system but unchanging, almost formless, as though the pieces had no beginning and no end but were simply extracts taken haphazardly from a ceaseless and ever-flowing score” (Dubuffet, 1961). What seems particularly interesting is his focus on ambiance as one of the factors influencing composition. This is the area which I will pursue in my compositional explorations during this creative research project.

Jean Dubuffet: side B of “Musique Brut” (uploaded by lutheranheresy on 9 September 2011)

[transforming-visual-to-audio]

Further notion on creative unconscious and intuition

PetersonJust as Jung describes the broad artistic fantasy, it seems like most of the explorations of creative process have one crucial characteristic in common —they refer to connection and manifestation of an unconscious. While the word choice may vary, the idea of connecting with an unconscious stays the same and this is what I seek to explore.

The unconscious part of the creative process has been a subject of interest to many scientists, philosophers, and artist themselves. As Erich Neumann writes in his book „Art and The Creative Unconscious” the creative transformation is often based on ‚possessions and projections, in other words on factors of highly dubious origins’ (Neumann, 1959, p.166). According to him, since Renaissance, the creative principle has been evolving and it no longer lays in cultural collective symbolism and canon but in the individual himself. Moreover, the creative principle has its source in ‚the deepest and darkest corner of his unconscious, and in what is best and highest in his consciouses, that we can comprehend it only as the fruit of his whole existence’ (Neumann, 1959, p.169). To Neumann, the fundament of creative process lays in a progression from unconscious to consciousness, what for him means: ‚immersing […] in worlds of archaic primitive symbolism’ (Ibid.) He then explains how such archaic and regressive model is perceived as unwanted and should be suppressed with development. An example he gives „so-called scientific view” including psychoanalysis. (Ibid.) He claims that perceiving such symbolism just as an initial stage of the development of rational consciousness is dangerous and insufficient. For him, the capability of one’s to create symbols can only be understood if we recognize that:

‚symbols reflect a more complete reality than can be encompassed in the rational concepts of consciousness’  (Neumann, 1959, p.170)

Perhaps surprisingly, such thinking can also be found in Lois Oppenheim’s book „Psychoanalysis and the Artistic Endeavor”. While the author recognizes the importance of psychoanalysis in order to understand a creative process, for him the biggest insight to this ‚capacity for image and symbol formation’ (Oppenheim, 2015, p.3) is available through conversations with artists themselves. Therefore, I will seek to bring some of the artists' thoughts and ideas on a creative process: how unconscious mediates their work and vice versa.

Francis Bacon’s creative process is characterizes by constant interplay of conscious and unconscious. Bacon himself explains:

‘I have an idea, start working and it evaporates. If it goes well something will begin to crystallize.Putting paint on, wiping it off…sometimes the shadows left will lead to another image.Something is only willed when the unconscious thing has begun to arise on which your will can be imposed. It is a dialogue…the paint is suggesting things to you…it’s a constant exchange.’ (Bacon, cited in Marks 2014, p.16)  

Composer Steve Reich explains the crucial role of intuition in his musical practices:

‚The choice of pitch and timbre in my music has always been intuitive. Even the choice of rhythmic structure is finally intuitive. In fact, although there is always a system working itself out in my music, there would be no interest in the music if it were merely systematic.... The truth is, musical intuition is at the rock bottom level of everything I've ever done.’ (1968) 

Marcel Duchamp, a pioneer of 20th-century Dada movement, gave a speech dedicated to  a creative process. Duchamp compares an artist to a ‚mediumistic being’ who through a ‚the labyrinth beyond time and space, seeks his way out to a clearing.’ (1957 cited in Peterson&Sanouillet 1975, p. 138) He then accentuates the medium metaphor and its connection to consciousness:

‚If we give the attributes of a medium to the artist, we must then deny him the state of consciousness on the esthetic plane about what he is doing or why he is doing it. All his decisions in the artistic execution of the work rest with pure intuition and cannot be translated into a self-analysis, spoken or written, or even thought out.’ (Ibid.)

According to Duchamp, one of the very important factors during a creative act is not only intuition but also intention. The artist explains that during the creative act, one enters a journey from intention do realization, through what he calls ‚a chain of totally subjective reactions.’ (Ibid.)

According to him, such journey is accompanied by series of struggles, efforts, decisions what eventually leads to the difference between the initial intention and final realization. Thus, a missing link between the intention and realization is what he calls a personal ‚art coefficient’ which contains the work. In his exact words, the ‘art coefficient’ is:

‚like an arithmetical relation between the unexpressed but intended and the unintentionally expressed.’ (Ibid.)

What I will then seek to explore by automatic drawing is one side of the ‚art coefficient’ — to investigate what is unintentionally expressed.

[transforming-visual-to-audio]

Psychological background of the project

‚The dynamic principle of fantasy is play, a characteristic also of the child, and as such it appears inconsistent with the principle of serious work. But without this playing with fantasy, no creative work has ever yet come to birth. The debt we owe to the play of the imagination is incalculable’ — C.G. Jung (1921/1990, par. 93)

As mentioned in the previous post, intuition is one of the main areas of exploration for this project, therefore, it is necessary to bring a broader context of this term. Intuition has been attempted to describe and explore for centuries and there are many definitions available. Perhaps the most widespread description of the unconscious, on the whole, is proposed by Swiss psychiatrists and psychologists — Carl Gustav Jung. To understand the intuition more, we first should look at his categorization of ‚psyche’. According to him, ‚psyche’ means: ‚the totality of all psychological processes, both conscious as well as unconscious’ (Jung 1921 cited in Jacobi 1952, p.588)

Therefore, the psyche is comprised of two spheres which are complementary yet opposite— consciousness and unconscious, both of them shared by our ego. (Jacobi, 1942, p.20). For the purposes of this project, I will focus mainly on the unconscious and its connection with creative processes. According to Jung, the unconscious is ‚relations to the ego, in so far as they are not sensed as such by the go’ (1921, p. 536). Furthermore, the unconscious can be also divided to: 

1. ‘PERSONAL UNCONSCIOUS’

2. ’COLLECTIVE UNCONSCIOUS’

The first one is described as: 'forgotten, repressed, subliminally perceived, thought, and felt matter of every kind’ (1921, p. 616) The later is depicted as: ‘the inherited possibility of psychical functioning in general, namely from the inherited brain structure’ (Jung, 1921, p.616). The consciousness arises from the unconscious. As Jacobi mentions Jung’s words, in pivotal life situations, our conscious decision depends on the unconscious. This shows the prominence of the unconscious in our daily lives and possible area of self-discovery.

To understand unconscious further, it is helpful to recognize four basic functions present in every individual: thinking, feeling, intuition, sensation. (Jacobi, 1942, p.25). While the first two remain rational, Jung categorizes both intuition and sensation as irrational functions. Sensation sees things they way they are. Intuition does so similarly, although less through our senses and more through ‚unconscious inner perception’ (Jacobi, 1952, p.27). Now, It is important to notice that intuition is just a part of four functions which altogether create the unconscious. Similarly, for Jung, intuition is just one of four elements responsible for artistic creativity. That is to say, the combination of four functions (thinking, feeling, intuition and sensation) and what is beyond them create ‚fantasy’. (Jacobi, 1942, p.28). By Paraphrasing Jung, Jacobi describes that the fantasy brings a very specific form of a manifesto in the psyche: ‚THE IMAGE’. The image depicts the material of the unconscious. That is to say, the image:

‚commutes the chaos of the unconscious contents into pictorialized manifestations, as they present themselves in dreams, in fantasies, in visions and, analogously to these, in every act of creative art.’ (Jacobi, 1942, p. 75)

As it appears, the image is one of the forms of playing with fantasy, of manifesting it. In this project, I want to focus on such on creating such play-space with unconscious fantasy, on bringing the chaos of the unconscious to consciousness through art. I believe that two mediums: audio and visual represent this connection of unconscious/consciousness - tangible/intangible. 

[transforming-visual-to-audio]

Project’s background

Intuition has played a very important role in my artistic experience as long as I remember. My artistic explorations didn’t start with music — as a kid, my main area of creative explorations was drawing. I would sit for hours drawing all of the different shapes, colours, creatures, disappearing in my imagination. Now, of course, this is not something unusual for a child, on the contrary, a child’s universal creativity is something that appears to vanish within ageing.(Anderson, 1959, p.44)

Personally, when I was around 13-years-old I made a shift from drawing to playing the guitar, and tough this practices differ a lot, my intuitive approach to both had a lot of similarities. Music became my main area of creative focus since adolescence yet I never fully abandoned visual arts. I believe that these two surfaces, when intermediated, open new avenues for explorations. These explorations are my main areas of focus for this project — I want to I want re-discover inner intuitional creativity by challenging myself and my on-going compositional processes and habits. Therefore, I want to discover new possible methods for composition by bridging the audio and visual. This is why I aim to create new intermedia work where one medium would not exist without the other.  

[transforming-visual-to-audio]

The main idea of this creative research project is to explore and understand transmediation process and compose a piece not emerging from musical principles. That is to say, I seek to explore a hybridization of visual and musical processes to allow for the creation of pieces that could not be created on their own. In order to achieve this transmediation, I will explore intuitional (automatism), analytical (sonification) and hybrid approaches.

I will neither attempt to proof nor disproof the presence of unconscious in my work. The unconscious related approach and lack of intention will rather act as a catalyst for discovering new compositional approaches. By creating purely coincidental visual stimuli I am depriving myself of any intentions and associations. Of course, it could be argued that the same lack of intention could be achieved by purely musical experimentations. Yet I would suggest that in the instance of this project pen or pencil used for drawings acts as a new medium in the process of approaching music differently. I want to create drawings which are not determined by any kind of mental guidance (reason, taste, morals) what could affect my compositional approach beforehand. Visual interplays with audio on one side of ‚art coefficient’ — they aim to discover what is unintentionally expressed. Psychologists and musician Patricia Skar calls such process an active imagination, in which she describes:

‚psychic space for new patterns of thinking emerges, and also that the new conscious mind is somehow ‚tuned’ by giving outer form to material that comes out of deep structural layers of the unconsciousness’ (Skar, 2002, p.632) 

Echoing Skar’s words, artist Charlotte Hug underlines the importance of intermediating artistic practices in order to reveal such new patterns of thinking:

„The aim is not an unconditional joining of drawing and music, but the recognition of visual and acoustic ways of thinking and qualities, which accordingly generate other ideas and other worlds. Nevertheless, at certain moments they both enter into an intense symbiosis.” (Charlottehug.ch, 2018)

Therefore, in the instance of my work, it is not about the drawings themselves but rather about what the drawings allow. Hence it is not about the resultant piece itself but rather about the process leading to it. The musical composition could not exist in its form without the relationship with drawings and this is what I seek to pursue. Similarly to Hug’s work, in my project the audiovisual relationship is not a linear but rather genealogical, and transcendental, it is a space where one medium determines another. I will not try to ‘sound’ the images but rather to use them us stimuli/source for the composition. Musical composition, therefore, acts as a non-intentional continuation of the intuitional drawing process. This process will, therefore, lack any previously preconception and will require new means of expression and much more unconscious approach.