Working On: 80Hz

“Using sonification, the data could become experiential and temporal.”

Thomas Wing-Evans

This architectural sound lab created by Thomas Wing-Evans, in collaboration with the DX Lab and sound designer Julian Wessels, is a sonification of 40 mesmerising artworks from the State Library of New South Wales’ Collection.

The State Library of NSW has set the ambitious goal of digitising approximately 20 million collection images and pages during the next decade. The generation of all of this data opens the door to new ways of interpreting the Library’s collection. Recipient of the 2017 DX Lab Fellowship, artist Thomas Wing Evans states that, “Digitisation is a brilliant method of preserving a moment in time, whilst also providing new ways to link different artefacts through data. Paintings no longer exist in isolation, but become networked through metadata… Now, in digital form they are able to be reinterpreted, translated and remixed as open-ended artworks.”

80Hz explores what it would look like for an audience to interact with a collection in a digital realm using sound as a conduit to exploration and in tern interpretation. To achieve this, we would extract data sets from each painting, drawing on their differences and similarities to extrapolate a set of rules and filters that could feed into a composition engine to create 40 unique data-driven compositions. To make this possible a bespoke program was created as well as a custom algorithm that could scales various sets of data and apply them to different elements within a composition. After a lengthy period of experimentation we had crafted an algorithm that we where satisfied represented the full spectrum of data in musical form that was intricate enough to be interesting, but not too abstract as to be unaccessible or alienating to a general audience. What we didn’t want to do was to just compose piece of music to accompany each painting because this would carry with it our own interpretations and emotional biases. Though listening to some of the pieces it is hard to imagine that some emotive cues have not been scripted, but in reality the algorithm merely looks at metadata gathered during the digital analysis of each painting and does not take emotion of any kind into account. As humans we are meaning makers, always looking for clues on how to interpret the world around us and it has been a truly rewarding experience seeing the unique perspectives people have in response to the compositions.

The final element of the experience was the element of play. We wanted to have some hidden gems for viewers to discover as a way of encouraging and rewarding exploration. A physical mechanism in the centre of the structure allowed visitors to cycle back and forth through the painting at their own pace. We crafted a dedicated program to create smooth, playable transitions between soundscapes with the use of RF ID tags lining the back of the reel of paintings. It is around these transition points between images that, if lingered on, makes way for some unexpected audio moments to occur based on the semi-random way the program manipulates the compositions. Around the structure, a 4.2 channel speaker set up lined the inside of the structure, enveloping the listener in a cocoon  of sound, a sonic refuge in the heart of the bustling city.

If you are interested in some of the tech behind this project there is a great article about it on the DX Lab’s Website