Hack the Audience @ MTF Berlin

CoSiMa participated at the Music Tech Fest in Berlin with a workshop ¬ę¬†Hack the Audience ¬Ľ¬†featuring the Soundworks framework. In two days, mai 26 and 27, we developed two performances in which the audience participates with their smartphones : MTF Orgy and¬†GrainField.¬†In both performances, the audience connects their smartphones to the CoSiMa¬†Wi-Fi network and visits a given webpage¬†to participate.

In¬†MTF Orgy, each participant controls the intensity and detuning of two harmonics¬†of a distributed additive synthesizer ‚Äď the Orgy organ ‚Äď by tilting their smartphone. The¬†lower harmonics are¬†generated on¬†the PA and the higher ones on¬†the participant’s mobile¬†devices. A musician on stage plays chords¬†on a MIDI keyboard that determine the fundamental frequencies. Other musicians can join the performance. At the MTF performance, we were accompanied by Steve Lawson on the bass.

In GrainField the smartphones enable the participants to play with the granular synthesis of 2 secs of sound recorded from a percussionist sitting in the middle of the audience (see images below). The system records every second 2 secs¬†of sound that are¬†send to the smartphones¬†of the audience so that the sound a¬†participant plays with changes every 8 seconds. The¬†sound generated by the participant’s¬†smartphones can be¬†perceived as a distributed granular echo of the percussionist’s performance without any¬†other¬†amplification.

In addition, we presented the CoSiMa project in a brief talk and performance with the audience playing birds and drops on their smartphones.


Electrosound @ EDF Foundation

CoSiMa is represented in the Electrosound exhibition at the EDF Foundation with two small installations. One documents the participative concert¬†Chlo√© ‚®Č Ircam¬†and the other is a downscaled version of Collective Loops.

The installation that refers to the participative concert we created with Chloé, is based on the same technical setup and music track as the Terminal installation and features 14 wall mounted smartphones. As in the concert and the Terminal installation, the public can connect to the installation to participate using their mobile devices.


Collective Loops is shown in a reduced version with eight smartphones mounted on desk and a screen that replaces the floor projection.


The exhibition¬†also included the MO ‚Äď Musical Objects that have been developed in the framework of one of CoSiMa’s predecessor projects, Interlude.


CoSiMa in ¬ę Le Future de la Musique ¬Ľ

The participative concert Chlo√© ‚®Č Ircam¬†and the interactive installation Terminal, both¬†created in the framework of CoSiMa, have been¬†presented in the documentary ¬ę¬†Le Future de la Musique ¬Ľ¬†on the television program¬†PLAN√ąTE+. The documentary includes¬†interviews with Chlo√© and Norbert Schnell as well as footage from a working session with Chlo√© and Ircam’s CoSiMa team in a studio at Ircam.

Here is the trailer of the episode that features the CoSiMa projects :

Workshop ¬ę Smartphones and Movement ¬Ľ @ ENSCI – Les Ateliers

In collaboration with, Jo√ęl Chevrier (professor of physics at the Grenoble Alpes University), Anne Dubos (anthropologist and transmedia artist), Lo√Įc Lobet (designer), and Florent Deloison (artist and developper), we proposed a¬†two-week workshop on¬†Smartphones and Movement¬†at the¬†ENSCI – Les Ateliers¬†design school.

The workshop united a group of about twenty students who worked in small groups on the design of different projets including performances, installations, and gadgets involving smartphones in motion, as well as concepts of controlling common smartphone applications through movements.

Here are some of the projets developed during the workshop.

Physical Player

Physical Player is a performance for three performers wearing mobile devices on their arm. The performers move in a large space interpreting trajectories and symbols marked on the floor. Their motion induces events and changes of sound and music environment rendered through loudspeakers distributed in the space. The mobile devices analyse the performers’ movements distinguishing different states like standing still, walking, running (fast and slow), jumping, and¬†somersault to control three different tracks of an interactive audio player.


Sorcières is a performance that integrates some of the interactive web audio applications we created at the beginning the CoSiMa project (see apps.cosima.ircam.fr/checks). During the performance, the performers invite the public to join their choreography and to produce sound with their smartphones.

Le petit avion

The project The Little Plane transforms a smartphone into a toy that makes sound when it is moved. In addition, the toy can analyse and display the motion data trajectories and can communicate them to a public display. Obviously, the plane could be a car, a boat, a train or anything else that makes sound when it moves.


The Scorpion web application transforms subtile motion and vibrations into sound. The applications has been used by the students to create a tactile sound installation.

The application has also been used in ¬ę¬†L‚ÄôAtelier tactile ¬Ľ,¬†a playground for little children created by Marion Voillot. In this scenario, the children search ‚Äď by carefully touching and listening ‚Äď for objects (i.e. mobile devices) that generate sound and light hidden in fabrics of different textures embedded into a wooden structure.

ūüĒó¬†Good Vibrations (3/3) : Le Smartphone et le Scorpion (article on the Scorpion project in French)

Interactive Storytelling

Another follow-up project of the ENSCI workshop was an application dedicated to little children created in collaboration with Marion Voillot. The application supports a scenario of storytelling where children accompany different parts of a story with different gestures that are translated into sound by smartphones they hold in their hands. The movements and sounds evoke actions and motion that occur in the story (i.e.¬†swimming, wading trough water, pushing aside the scrub, wind over a meadow, storm) and where chosen to stimulate the children’s imagination, but also to reinforce coordination, memory, and listening.

ūüĒó¬†Le jeune enfant, le num√©rique et l’√©cole demain¬†(article by Marion Voillot on her exploration of digital technology for little children)

Overexposure / Surexposition v.2

An interactive public installation with smartphones, Palais de Tokyo, Paris, February 2016

Surexposition/Overexposure (v.2) is an interactive work bringing together a public installation and a smart phone application. On an urban square, a large black monolith projects an intense beam of white light into the sky. Visible all over the city, the beam turns off and on, pulsating in way that communicates rigor, a will to communicate, even if we don‚Äôt immediately understand the signals it is producing. On one side of the monolith, white dots and dashes scroll past, from the bottom up, marking the installation with their rhythm: each time one reaches the top of the monolith, the light goes off, as if the marks were emptying into the light. On a completely different scale, we see the same marks scrolling across the smartphone screens of the people in attendance, interacting with the work, following the same rhythm. Here, it is the flash of the smartphones that releases light in accordance with the coded language. Returning to the very essence of Morse, the messages are then transformed into a sound composition, broadcast by the installation, as well as by the public‚Äôs smartphones. Because these are in fact messages that are being sent‚ÄĒin Morse code, from everyone, to everyone and to the sky‚ÄĒand that we can read thanks to the super-titling that accompanies the marks. Using a smartphone, anyone can send a message, saying what they think and therefore presenting themselves, for a few moments, to everyone, to a community sharing the same time, the same rhythm. And we can take the pulse of an even larger community‚ÄĒon the scale of the city and in real time‚ÄĒthrough a map of mobile phone network use, projected onto the ground or visualized via smartphone.

From an individual device (smartphone) the size of a hand to a shared format on the scale of the city, a momentary community forms and transforms, sharing a space, a pace, the same data, following a type of communication whose ability to bring together through a sensory experience is more important than the meaning of the messages it transmits or their destination, which is lost in the sky.

(Photos: Samuel Bianchini)

An Orange/EnsadLab (the laboratory of the √Čcole Nnationale sup√©rieure Sup√©rieure des Arts D√©coratifs, PSL Research University) project under the direction of Samuel Bianchini

in collaboration with Dominique Cunin (EnsadLab), Catherine Ramus (Orange Labs) and Marc Brice (Orange Labs), in the framework of a research partnership with Orange Labs, and Roland Cahen (sound design).

“Orange/EnsadLab‚ÄĚ partnership directors: Armelle Pasco, Director of Cultural and Institutional Partnerships, Orange and Emmanuel Mah√©, Head of Research, EnsAD

Project Manager (Orange): Abla Benmiloud-Faucher
IT Development (EnsadLab): Dominique Cunin, Oussama Mubarak, Jonathan Tanant
Graphic Design: Alexandre Dechosal (EnsadLab)
Sound Design: Roland Cahen
Sound engine development: Norbert Schnell (Ircam)
Voice samples recorded by choristers from Jazzalam
Lighting, Video Mapping and construction of the installation structure: idscènes
Mobile network data supply: Orange Flux Vision
Mobile network data processing: Cezary Ziemlicki (Orange)
SMS Server Development: Orange Applications for Business
Technical Assistant: Alexandre Saunier (EnsadLab)
Production Assistant: √Člodie Tincq

Production: Orange
Executive Production: EnsadLab

The research and development for this work were carried out in association with the research project Cosima (‚ÄúCollaborative Situated Media‚ÄĚ), coordinated by Ircam and sponsored by the Agence nationale de la recherche (ANR) and participate in the development of Mobilizing.js (http://www.mobilizing-js.net), a programming environment for mobile screens, conceived by EnsadLab, aimed at artists and designers.
This version of Surexposition benefits from a partnership with idscènes and Ircam under the aegis of the Cosima project.

Collective Loops @ Forum Workshops

A second version of the Collective Loops installation has been shown during the Ircam Forum Workshops on November 25 and 26, 2015.

The installation features a collaborative version of a step sequencer that uses the visitor’s smartphones to produce sound. The sequencer is graphically represented by a circle of 8 sectors projected on the floor. The sectors light up in a clockwise motion following the beats of the sequence.

When the players connect to the installation through a web page, they choose an available sector, and thus, their step in the loop. The players can control the sounds (i.e. notes in a melody or bass line and percussion sounds) that are played on their smartphone at the corresponding beat of the sequence through a simple graphical interface on their smartphones. The selected sounds are also displayed in the corresponding sector of the circle on the floor.

Positioned around the circle, the players collaborate on creating melodies and rhythm patterns rendered through their smartphones.

The application uses a first complete version of the CoSiMa platform entirely based on web standards.

Design and development :
Ircam: Norbert Schnell, Jean-Philippe Lambert, Benjamin Matuszewski, Sebastien Robaszkiewicz
Orbe: Xavier Boissarie, Florent Dubois, Gregory Cieslik, Tomek Jarolim, Quentin Levigneron
EnsadLab: Samuel Bianchini, Dominique Cunin, Oussama Mubarak, Jonathan Tanant
ID Scènes: Christophe Aubry, Fabrice Auchere
NoDesign: Jean-Louis Frechin, UroŇ° Petrevski
ESBA TALM: Christophe Domino

Terminal @ Paris Musique Club

Terminal is an interactive installation that has been created in collaboration with Chloé and the Scale collective for the Paris Musique Club. The installation will be shown from October 24, 2015 to January 31, 2016 at the Gaité Lyrique.

The project transposes the musical elements and mobile interactions of the Chlo√© ‚®Č Ircam concert into the situation of an exhibition.
The installation features a looped 15-minutes 4-channel music track staged in a 7-meters corridor with 21 smartphones aligned along the wall and luminous lines running on the floor.

Similar as in the concert, visitors can connect to the installation with their mobile devices to participate. At given passages of the music track, the participants are invited to play sound with touch and motion interfaces that appear on their mobile device. The graphical animations and sound of their device are echoed by one of the smartphones on the wall.


Every now and then, waves of¬†sound textures appear on the participants’ mobile devices. In addition, visitors can use a wall-mounted tablet to distribute¬†sound textures over¬†the smartphones on the wall. The light on the floor reacts on the music as well as the visitors’ interactions with the tablet.

Chloé × Ircam @ Nuit Blanche 2015

We did¬†it again. After the Fete de la musique,¬†the Chloe¬†√ó Ircam¬†concert came to the¬†Nuit Blanche¬†at the Ga√Įt√© Lyrique invited by I Love Transmedia.

This video summarizes the concert from a rather technical point of view.

Like at the¬†Fete de la musique,¬†the audience participates in this concert by connecting their smartphones to the¬†local Wi-Fi network¬†Chloe¬†√ó Ircam and by opening¬†the¬†web page chloe.ircam.fr¬†in their browser. Once connected, the participants¬†are¬†asked to indicate their approximate position on map of the concert space. During the concert, Chlo√© can move sounds over the¬†audience’s smartphones ‚Äď using four tablets¬†integrated into her setting ‚Äď and let appear dedicated sound interfaces on the touchscreens. The concert starts and ends with everybody playing with Chlo√©’s whispering voice.

More images of the concert are published on Chlo√©’s web site.


Murmures Urbains @ Chalon dans la rue 2015

CoSiMa has participated with Murmures Urbains at 2015 edition of the festival Chalon dans la rue from June 24 to 26.

The¬†Orbe collective has presented three experimental scenarios of augmented soundwalks. The¬†participants equipped with a smartphone are invited to experience an augmented audio reality that reacts on their position, trajectory and movement¬†(using GPS, BTLE beacons and motion sensors).¬†Each scenario proposes a different narrative and leads the participants on different possible trajectories through the same district of Chalon-sur-Sa√īne. The trajectories take between 30 minutes and 2 hours depending on the participants’ preference and their engagement with the proposed activities.

The trajectories of all participants have been recorded and visualized on a screen at the arrival point where the team invited the participants to a debriefing of their experience.