collective sound check

Open House @ IRCAM

During IRCAM’s open house on June 6th, CoSiMa presented two different projects, Collective Loops and Woodland.

Collective Loops

Collective Loops is a collaborative version of an 8-step loop sequencer. When visitors access the webpage of the installation with their smartphone, they are automatically assigned to an available step in the sequence loop, and their smartphone plays a sound when it is their turn. The participants control the pitch of the sound through the inclination of their smartphones. The participants are invited to collaboratively create a melody of 8 pitches that circulates in a steady tempo over their smartphones.

A circular visualization of the sequencer is projected on the floor. The projection consists of a circle divided in 8 sections that light up in counterclockwise circular movement synchronized with the sounds emitted by the smartphones. Each section of the projection is further divided into 12 radial segments that display the pitch of the corresponding sequence step (i.e. controlled through the inclination of the participants smartphone).

The 8 first participants who connect to the sequencer have a celesta sound, the 8 following can play with a drum kit, the 8 last have a bass sound. All together, 24 players can create complex rhythmic and melodic patterns.

Woodland

Woodland is a very early stage prototype that aims at explaining how natural audio effects (such as reverb) are created in the natural environment. For this, we create a setting where each participant is a tree in a forest. At some point, a designated player “throws a sound” in the forest by swinging his / her smartphone upwards. After a few seconds of calculations, the sound falls on one tree; then we hear the first wave of resonances when the sound reaches the other trees; and so on recursively until the sound ultimately vanishes.

In order to make people understand what is going on, we can control several parameters of the simulation such as the speed of sound in the air, the absorbance of the air, the type of sound (with a hard or soft attack), etc. That way, if we set the parameters to be similar to the natural setting, we hear the same reverb as we would hear in a forest. But if for example we slow down the speed of sound, we can hear a very slow version of how this natural reverb is built, hearing each echo one by one.

This very first prototype was very promising, and further developments might include a visualization on the floor of the different sounds that bounce from trees to trees to create that reverb effect.

Web Audio Now! @ Best of Web 2015

When Cyril Balit participated to the CoSiMa demonstration at Paris Face Cachée, he asked us if we would be willing to present our work to the Best of Web 2015 conference in Paris, which is a compilation of the best talks of 8 Parisian web meet-ups.

So there we went! Together with the WAVE project — the IRCAM Web Audio library on which most of the things we do is based on, see also wavesjs on GitHub) —, we presented our ongoing research and projects. In particular, we took advantage of this event to test a new collaborative experience that would premiere at Fête de la musique (see Chloé × Ircam). With more that 130 connections, it confirmed that we were on the right track for the show! We also got a lot of positive feedback from the JS community on the technologies we are developing.

The slides of the presentation are available here.

Rone : Créatures & Cie @ Palais de Tokyo

Rone was invited to unveil the universe he created for his latest album Créatures at the Palais de Tokyo (Le Point Perché). Along with illustrations, photographs, binaural experiences and video games, CoSiMa presented Créatures & Cie – Collective Sound Check, a spontaneous collective performance that enables the audience to play with Rone’s creatures. By simply opening a web page, the visitors of the exhibition can discover a novel way of exploring Rone’s musical universe and fill in the space with his sound creatures.

http://cosima.ircam.fr/creatures-et-cie (from a smartphone)

This performance is a first step towards a new generation of interactive musical experiences that Rone is developing in collaboration with IRCAM.

Collective Sound Check @ Paris Face Cachée

On February 6th, 7th and 8th, La Ville de Paris and the À Suivre association organized the 4th edition of Paris Face Cachée, which aims at proposing original and off-the-wall ways to discover the city. The CoSiMa team led the workshops Expérimentations sonores held at IRCAM on February 7th.

Three groups of 24 participants could test the latest web applications we developed. The audience first tried a few soundscapes (Birds and Monks) to get familiar with the sound-motion interactions on their smartphones, and to learn how to listen to each other while individually contributing to a collective sonic environment.

In the second part of the workshop, we proposed the participants to take part in the Drops collective smartphone performance. While the soundscapes also work as standalone web applications (i.e. they do not technically require other people to play with), Drops is inherently designed for a group of players, where the technology directly supports the social interaction. The players can play a limited number of sound drops that vary in pitch depending on the touch position. The sound drops are automatically echoed by the smartphones of other players before coming back to the player, creating a fading loop of long echoes until they vanish. The collective performance is accompanied by a synchronized soundscape on ambient loudspeakers.

The performance is strongly inspired by the mobile application Bloom by Brian Eno and and Peter Chilvers.

Below are a few pictures from the event.

CoSiMa @ WAC’15

At the first international Web Audio Conference (WAC’15), CoSiMa presented three pieces of work.

Collective Sound Checks (poster)

Just like at TEI’15 the week before, we presented our work on the Collective Sound Checks through the poster you can see below. Quite a lot of people gathered at our booth during the demo session to play with the web apps and create spontaneous collective performances.

Soundworks (paper & poster)

We presented a draft of the Soundworks library (which has evolved quite a lot since then): Soundworks is a Javascript framework that enables artists and developers to create collaborative music performances where a group of participants distributed in space use their smartphones to generate sound and light through touch and motion.

In particular, we used Soundworks to build the Drops collective performance (see below). You can read the WAC paper here, or have a look at the Github repository for more up-to-date information. Finally, you’ll find the WAC poster below.

Drops (performance)

Finally, we presented the first public representation of Drops, a collective smartphone performance built with Soundworks. Drops is strongly inspired by the mobile application Bloom by Brian Eno and and Peter Chilvers, and transposes it into a collaborative experience: each participant can only play a single sound (i.e. a single pitch), whose timbre can vary depending on the touch position. Together, the players can construct sound sequences (i.e. melodies) by combining their sounds. The sounds are repeated in a fading loop every few seconds until they vanish. Players can clear the loop by shaking their smartphones. The sounds triggered by one player are automatically echoed by the smartphones of other players. The collective performance on the smartphones is accompanied by a synchronized soundscape on ambient loudspeakers. This first Drops representation gathered around 60 players at the WAC.

CoSC: WWRY:R

We Will Rock You: Reloaded

This mobile web application has been developed in the context of Collective Sound Checks with the Studio 13/16 at the Centre Pompidou.

This application allows a group of players to perform Queen’s song “We Will Rock You” with a set of simple instruments and to create their own versions of the song. The players can choose between drums, voice solo, choirs, Freddy Mercury’s voice fill-ins (‘sing it’), a guitar power chord, and the final guitar riff.

While most of the instruments trigger segments of the original recordings when striking with the device in the air, the power chord and guitar riff resynthesize guitar sounds through granular synthesis.

The application has been published here (requires a mobile device under iOS 6 or later, or Android 4.2 or later with Chrome 35 or later).

CoSC: Shaker

This mobile web application has been developed for the Collective Sound Checks that took place in the Studio 13/16 at the Centre Pompidou.

In this scenario, players can record arbitrary percussive sounds (with their voice or using props) using a microphone (and a foot pedal). Once the recording is finished, the players can load the recorded sound on their mobile devices and perform it by shaking the devices. All the devices are beat synchronized to a steady tempo (16th beats on 100 BPM) so that multiple players can easily perform together.

Each sound recording is analysed on the server and segmented into percussive elements that are classified by their intensity. On the mobile device, the concatenative synthesizer generates a sound on each beat. Each sound is selected according to the device motion intensity: the synth plays soft sound segments when the player shakes the device softly, and louder segments when the player shakes the device more vigorously.

The players are encouraged to record phrases with percussive elements of a wide dynamic range. They can experiment with different sound recordings and create an ensemble by recording complementary materials.

CoSC: Matrix

This mobile web application has been developed in the context of Collective Sound Checks with the Studio 13/16 at the Centre Pompidou.

The players are sitting on a grid (for instance, 3 rows by 4 columns, for a total of 12 people). Their mobile devices form a matrix of screens and loudspeakers that are used to spatialize sound and light.

For now, the Matrix is performed by one player at a time: a representation of the matrix appears on the screen of a player who becomes the performer: by moving his finger on the matrix (on the screen), he controls from which smartphone(s) the light and sound come from in the real world. (The sound changes with the speed of the finger trajectory.) That way, he remotely uses the other people’s instruments. After a fixed time, another player takes over the control of the sound and light, and becomes the new performer.

The video below gives an idea of the technical setup. While the players are usually seated at a distance of 1 or 2 meters from each other, the smartphones are spaced by a few centimeters only for the purpose of this video.

The sound is generated locally on the mobile devices that are connected to a Web Socket server (using node.js and socket.io). The server receives the position from the performer’s device and controls the sound generators of all devices of the matrix.

CoSC Web Applications

For the Collective Sound Checks with the Studio 13/16 at the Centre Pompidou we have developed a series of web applications for mobile devices. The applications are based on web technologies such as HTML 5, JavaScript, Web Audio API, and Web Sockets.

The first set of applications we developed are a few gadgets that produce sound depending on the device’s motion. The gadgets can be played individually or with a group of player and allow for exploring different, techniques, sound materials, and metaphors. The drone, birds, monks and the rainstick are described below.

In addition to these gadgets we have experimented with collaborative scenarios that are described in separate posts:

WWRY:R (We Will Rock You: Reloaded)
Shaker
Matrix

The gadgets and We Will Rock You: Reloaded application have been published at http://cosima.ircam.fr/checks (the applications work on mobile devices and require at least iOS 6 or Android 4.2).

Drone

The drone reacts on the device rotation and responds with the amplitude and frequency modulation of a set of oscillators generating a bass drone. Strongly shaking the device generates an electric sound synthesized through granular synthesis.

Birds

Birds is a collection of bird sounds that are played by jiggling the device. Each player can try different bird calls. Two or more players can communicate through tweeting and create a forest-like atmosphere of distributed bird sounds answering to each other.

Monks

Monks features a short extract of the song “Early Morning Melody” from Meredith Monk’s Book of the Days and the recording of a Tibetan chant. Both extracts are performed through granular synthesis by tilting the device sidewards. A group of players can form a choir.

Rainstick

The rainstick is based on sound materials that have been created by Pierre Jodlowski and also used for the audio visual installation Grainstick produced at Ircam in 2010. The player has to hold your device horizontally and tilt it up and down like a rainstick to produce sound.

Collective Sound Checks

An important aspect of CoSiMa is the experimentation of user scenarios, technologies, and content developed in the framework of the project with a community of users.

« Collective Sound Checks » are regular events that allow us to try out new developments with a larger number of users and to validate technological, aesthetic, and social hypotheses of our work. Each event proposes different experiences inviting users to play music together, or a game, or to discover augmented reality spaces.

The first CoSiMa Collective Sound Checks have been conducted in collaboration with the Studio 13/16.*

A first series of workshops a the Studio 13/16 happened in spring 2014 on May 14, May 28 and June 14 (for the Open House at Ircam). A second series followed in Fall/Winter 2014 on October 1, October 15, November 5, and December 17.

We developed a series of web applications for these sessions. A selection of these applications for smartphones is online at cosima.ircam.fr/checks (please visit from a smartphone).

* Studio 13/16 sur facebook.