Workshop « Smartphones and Movement » @ ENSCI – Les Ateliers

In collaboration with, Joël Chevrier (professor of physics at the Grenoble Alpes University), Anne Dubos (anthropologist and transmedia artist), Loïc Lobet (designer), and Florent Deloison (artist and developper), we proposed a two-week workshop on Smartphones and Movement at the ENSCI – Les Ateliers design school.

The workshop united a group of about twenty students who worked in small groups on the design of different projets including performances, installations, and gadgets involving smartphones in motion, as well as concepts of controlling common smartphone applications through movements.

Here are some of the projets developed during the workshop.

Physical Player

Physical Player is a performance for three performers wearing mobile devices on their arm. The performers move in a large space interpreting trajectories and symbols marked on the floor. Their motion induces events and changes of sound and music environment rendered through loudspeakers distributed in the space. The mobile devices analyse the performers’ movements distinguishing different states like standing still, walking, running (fast and slow), jumping, and somersault to control three different tracks of an interactive audio player.

Sorcières

Sorcières is a performance that integrates some of the interactive web audio applications we created at the beginning the CoSiMa project (see apps.cosima.ircam.fr/checks). During the performance, the performers invite the public to join their choreography and to produce sound with their smartphones.

Le petit avion

The project The Little Plane transforms a smartphone into a toy that makes sound when it is moved. In addition, the toy can analyse and display the motion data trajectories and can communicate them to a public display. Obviously, the plane could be a car, a boat, a train or anything else that makes sound when it moves.

Scorpion

The Scorpion web application transforms subtile motion and vibrations into sound. The applications has been used by the students to create a tactile sound installation.

The application has also been used in « L’Atelier tactile », a playground for little children created by Marion Voillot. In this scenario, the children search – by carefully touching and listening – for objects (i.e. mobile devices) that generate sound and light hidden in fabrics of different textures embedded into a wooden structure.

🔗 Good Vibrations (3/3) : Le Smartphone et le Scorpion (article on the Scorpion project in French)

Interactive Storytelling

Another follow-up project of the ENSCI workshop was an application dedicated to little children created in collaboration with Marion Voillot. The application supports a scenario of storytelling where children accompany different parts of a story with different gestures that are translated into sound by smartphones they hold in their hands. The movements and sounds evoke actions and motion that occur in the story (i.e. swimming, wading trough water, pushing aside the scrub, wind over a meadow, storm) and where chosen to stimulate the children’s imagination, but also to reinforce coordination, memory, and listening.

🔗 Le jeune enfant, le numérique et l’école demain (article by Marion Voillot on her exploration of digital technology for little children)

Collective Loops @ Forum Workshops

A second version of the Collective Loops installation has been shown during the Ircam Forum Workshops on November 25 and 26, 2015.

The installation features a collaborative version of a step sequencer that uses the visitor’s smartphones to produce sound. The sequencer is graphically represented by a circle of 8 sectors projected on the floor. The sectors light up in a clockwise motion following the beats of the sequence.

When the players connect to the installation through a web page, they choose an available sector, and thus, their step in the loop. The players can control the sounds (i.e. notes in a melody or bass line and percussion sounds) that are played on their smartphone at the corresponding beat of the sequence through a simple graphical interface on their smartphones. The selected sounds are also displayed in the corresponding sector of the circle on the floor.

Positioned around the circle, the players collaborate on creating melodies and rhythm patterns rendered through their smartphones.

The application uses a first complete version of the CoSiMa platform entirely based on web standards.

Design and development :
Ircam: Norbert Schnell, Jean-Philippe Lambert, Benjamin Matuszewski, Sebastien Robaszkiewicz
Orbe: Xavier Boissarie, Florent Dubois, Gregory Cieslik, Tomek Jarolim, Quentin Levigneron
EnsadLab: Samuel Bianchini, Dominique Cunin, Oussama Mubarak, Jonathan Tanant
ID Scènes: Christophe Aubry, Fabrice Auchere
NoDesign: Jean-Louis Frechin, Uroš Petrevski
ESBA TALM: Christophe Domino

Terminal @ Paris Musique Club

Terminal is an interactive installation that has been created in collaboration with Chloé and the Scale collective for the Paris Musique Club. The installation will be shown from October 24, 2015 to January 31, 2016 at the Gaité Lyrique.

The project transposes the musical elements and mobile interactions of the Chloé ⨉ Ircam concert into the situation of an exhibition.
The installation features a looped 15-minutes 4-channel music track staged in a 7-meters corridor with 21 smartphones aligned along the wall and luminous lines running on the floor.

Similar as in the concert, visitors can connect to the installation with their mobile devices to participate. At given passages of the music track, the participants are invited to play sound with touch and motion interfaces that appear on their mobile device. The graphical animations and sound of their device are echoed by one of the smartphones on the wall.

guiro-interfaces

Every now and then, waves of sound textures appear on the participants’ mobile devices. In addition, visitors can use a wall-mounted tablet to distribute sound textures over the smartphones on the wall. The light on the floor reacts on the music as well as the visitors’ interactions with the tablet.

Chloé × Ircam @ Nuit Blanche 2015

We did it again. After the Fete de la musique, the Chloe × Ircam concert came to the Nuit Blanche at the Gaïté Lyrique invited by I Love Transmedia.

This video summarizes the concert from a rather technical point of view.

Like at the Fete de la musique, the audience participates in this concert by connecting their smartphones to the local Wi-Fi network Chloe × Ircam and by opening the web page chloe.ircam.fr in their browser. Once connected, the participants are asked to indicate their approximate position on map of the concert space. During the concert, Chloé can move sounds over the audience’s smartphones – using four tablets integrated into her setting – and let appear dedicated sound interfaces on the touchscreens. The concert starts and ends with everybody playing with Chloé’s whispering voice.

More images of the concert are published on Chloé’s web site.

 

Murmures Urbains @ Chalon dans la rue 2015

CoSiMa has participated with Murmures Urbains at 2015 edition of the festival Chalon dans la rue from June 24 to 26.

The Orbe collective has presented three experimental scenarios of augmented soundwalks. The participants equipped with a smartphone are invited to experience an augmented audio reality that reacts on their position, trajectory and movement (using GPS, BTLE beacons and motion sensors). Each scenario proposes a different narrative and leads the participants on different possible trajectories through the same district of Chalon-sur-Saône. The trajectories take between 30 minutes and 2 hours depending on the participants’ preference and their engagement with the proposed activities.

The trajectories of all participants have been recorded and visualized on a screen at the arrival point where the team invited the participants to a debriefing of their experience.

Chloé × Ircam @ Fête de la musique 2015

The French Ministère de la culture et de la communication asked IRCAM to imagine a participative concert for the Fête de la musique ’15, whose theme was « Vivre ensemble la musique » (“To live music together”). We partnered with Chloé in order to design the interactive live experience Chloé × Ircam. During that experience, Chloé alternates between moments when she plays alone — partially distributing sound on the audience’s smartphones —, and moments when she leaves room for the audience to play with her using their smartphones, thus enabling a musical dialog between her and the audience.

The concert took place on June 21st at the Jardin du Palais-Royal in Paris. At the beginning of the concert, participants are invited to join a WiFi network and connect to a URL (chloe.ircam.fr). After they indicate their positions in the venue thanks to a simple interface, the experience can begin. In addition to her usual live electronics setup, Chloé has four tablets where each participant shows up as a circle at the indicated position. Touching these circles she can play different sound textures on the participants’ smartphones. When moving her fingers over the touch screens the sound textures move over the space of the audience. On an additional tablet, Chloé can enable four different interfaces (i.e. simple instruments) on the participants’ devices that they can play by touching the screen and shaking the device.

The preparation of this project took three months, during which we made two live tests, one at IRCAM in the beginning of May with around 30 colleagues and friends, and one at the Centre Pompidou on June 9 with over 150 participants.

Open House @ IRCAM

During IRCAM’s open house on June 6th, CoSiMa presented two different projects, Collective Loops and Woodland.


Collective Loops

Collective Loops is a collaborative version of an 8-step loop sequencer. When visitors access the webpage of the installation with their smartphone, they are automatically assigned to an available step in the sequence loop, and their smartphone plays a sound when it is their turn. The participants control the pitch of the sound through the inclination of their smartphones. The participants are invited to collaboratively create a melody of 8 pitches that circulates in a steady tempo over their smartphones.

A circular visualization of the sequencer is projected on the floor. The projection consists of a circle divided in 8 sections that light up in counterclockwise circular movement synchronized with the sounds emitted by the smartphones. Each section of the projection is further divided into 12 radial segments that display the pitch of the corresponding sequence step (i.e. controlled through the inclination of the participants smartphone).

The 8 first participants who connect to the sequencer have a celesta sound, the 8 following can play with a drum kit, the 8 last have a bass sound. All together, 24 players can create complex rhythmic and melodic patterns.


Woodland

Woodland is a very early stage prototype that aims at explaining how natural audio effects (such as reverb) are created in the natural environment. For this, we create a setting where each participant is a tree in a forest. At some point, a designated player “throws a sound” in the forest by swinging his / her smartphone upwards. After a few seconds of calculations, the sound falls on one tree; then we hear the first wave of resonances when the sound reaches the other trees; and so on recursively until the sound ultimately vanishes.

In order to make people understand what is going on, we can control several parameters of the simulation such as the speed of sound in the air, the absorbance of the air, the type of sound (with a hard or soft attack), etc. That way, if we set the parameters to be similar to the natural setting, we hear the same reverb as we would hear in a forest. But if for example we slow down the speed of sound, we can hear a very slow version of how this natural reverb is built, hearing each echo one by one.

This very first prototype was very promising, and further developments might include a visualization on the floor of the different sounds that bounce from trees to trees to create that reverb effect.

Web Audio Now! @ Best of Web 2015

When Cyril Balit participated to the CoSiMa demonstration at Paris Face Cachée, he asked us if we would be willing to present our work to the Best of Web 2015 conference in Paris, which is a compilation of the best talks of 8 Parisian web meet-ups.

So there we went! Together with the WAVE project — the IRCAM Web Audio library on which most of the things we do is based on, see also wavesjs on GitHub) —, we presented our ongoing research and projects. In particular, we took advantage of this event to test a new collaborative experience that would premiere at Fête de la musique (see Chloé × Ircam). With more that 130 connections, it confirmed that we were on the right track for the show! We also got a lot of positive feedback from the JS community on the technologies we are developing.

The slides of the presentation are available here.

Rone : Créatures & Cie @ Palais de Tokyo

Rone was invited to unveil the universe he created for his latest album Créatures at the Palais de Tokyo (Le Point Perché). Along with illustrations, photographs, binaural experiences and video games, CoSiMa presented Créatures & Cie – Collective Sound Check, a spontaneous collective performance that enables the audience to play with Rone’s creatures. By simply opening a web page, the visitors of the exhibition can discover a novel way of exploring Rone’s musical universe and fill in the space with his sound creatures.

http://cosima.ircam.fr/creatures-et-cie (from a smartphone)

This performance is a first step towards a new generation of interactive musical experiences that Rone is developing in collaboration with IRCAM.

Rone : Creatures & Cie