All posts by nchandol

Story Telling Space


Story Telling Space

This research project is an in depth investigation for the realization of a system, able to combine gesture and vocal recognition for interactive art, live events and speech based expressive applications. Practically, we have created a flexible platform for collaborative and improvisatory storytelling combining voice and movement. Our work advances both conceptual and technical research relating speech, body, and performance using digital technologies and interactive media.

We create an immersive audio-visual Story Telling Room that responds to voice and sound inputs. The challenge is to set up an efficient speech feature extraction mechanism using as good microphone conditions as possible. Leveraging the TML’s realtime media choreography framework, we map speech to a wide variety of media such as animated glyphs, visual graphics, light fields and soundscapes. Our desiderata for mapping speech prosody information centre on reproducibility, maximum sensitivity, and nil latency. Our purpose is not to duplicate but to supplement and augment the experience of the story that is being unfolded with a performer and audience in ad hoc, improvised situations using speech and voice for expressive purposes.



We are exploring the possibilities of Natural Language Processing in the context of live performance. As the performer speaks the system analyzes the spoken words and with the help of the Oxford American Writer’s Thesaurus (OAWT), each semantically significant lexical unit initiates its own semantic cluster.

As the story is being unfold by the performer the environment is shifting from a state to another according to the censoring data that occur from the system analysis. Furthermore, we are exploring the possibilities of transcribed text from spoken utterances. The spoken words of the performer already burry a communicative value as they already have attached to them a semantic component that has been evolved and transform through out the history of language. The objective of this prototype is to see what happens when light, imagery and texture is added in the text and how it is perceived by the performer.

The pieces of text when encountered as audio-visual forms animated by quasi- intelligent dynamics in digital media become widely perceived as animate entities. People tent to regard animated glyphs as things to be tamed or played with rather than a functional and abstract system of communicative symbols.

[youtube][/youtube] [youtube][/youtube]


We have create two stand-alone Java application that perform the Speech Recognition and Speech Analysis tasks. For the mapping techniques we have used partially the already existing ozone state engine and we have create a new state engine in MAX/ MSP for better results in the context of improvisatory story telling


Nikolaos Chandolias, real-time Speech Recognition & Analysis, System Design
Jerome DelaPierre, real-time video
Navid Navab real-time sound
Julian Stein, real-time lights

Michael Montanaro, choreographer/director
Patricia Duquet, Actress
Sha Xin Wei, Topological Media Lab
Jason Lewis, Obx Labs


Story Telling Space Documentation


Jennifer Spiegel

Jennifer Spiegel is a post-doctoral fellow in the department of Art History and Communications at McGill and teaches in the Department of the Theatre at both Concordia and McGill. She holds a PhD in Cultural Studies from Goldsmiths College, University of London. Her research focuses on performance and theatricality in environmental communication and social activism, and the politics of aesthetics across media. She is also interested in practice-based research concerning collective processes of creation in physical theatre, social circus and the possibilities afforded by experiments in the collective movement of human and non-human bodies.

Areas of interest include: Contemporary social and political theory; radical theatre and
performance; community-based performance; critical theories of performance and media; ecological thought; social movements; grass roots media

Zohar Kfir

Zohar Kfir is a Montréal-based artist working with video and interactive installation. She is a graduate of NYU’s ITP program (2002) and Concordia’s MFA program (2011). Her artistic practice focuses on the dynamic between growth and decay and the paradox of creating digital representations of natural phenomena. Zohar has shown her work internationally in galleries and video festivals; at the TML she has worked on prototyping electronics, documentation and assisting on various research projects.

In the past 3 years Zohar has worked mostly with the Memory-place group- meetings, assisting in research, prototyping electronics and documentation of the various experiments. She will be working on Tirtza Even’s project Failing.

JoDee Allen

JoDee Allen (BFA Contemporary Dance) is a SIP Masters student researching street dance,video games and real-time video. She is a part-time professor in the Contemporary Dance faculty and is the co-artistic director and founding member of the dance company, Solid State Breakdance (2000 – 2011). JoDee is currently involved in movement-based research at the TML, specifically the Gesture Bending project.

Jérôme Delapierre

Jérôme Delapierre is a visual artist and interactive designer working in Montreal, studied Computation Arts and Interactive design at Concordia University as well as Contemporary Arts and new media at IMUS University in France. Currently the artistic director of Active media inc and Anartistic, and a freelance visual designer and researcher at Topological Media Lab and Alkemie Atelier. He has collaborated with differents artists and researchers, like Pk langshaw, Sha Xin Wei, Michael Montanaro and Jean Derome, and have been presented at festivals and events in various countries. His research is based on the relationship between human and technology and non linear interactivity, focusing on the experiences of urban social behavior. He is interested in new ways to create visual sets and environments by exploring eclectic projections techniques. Jérôme work on responsive video, interactive installations, performances and scenography.

Lina Dib

Lina Dib is an artist and anthropologist associated with the Topological Media Lab in Montreal and TX/RX labs in Houston. Her PhD (2013) in Anthropology from Rice University concerned the contemporary technologies of memory. Situated at the intersection of anthropology, contemporary art, and the social study of science, her work focuses on concepts of time, memory, the environment, and intimacy in the design of digital recording devices. She has received grants and awards from Canada’s Social Science and Humanities Research Council, AMIDA’s European training program, and Rice University’s Humanities Research Center. Recent publications include: “The Forgetting Dis-ease: Making Time Matter” (2012), “Of Promises and Prototypes: The Archeology of the Future” (2010) and “Memory as Concept in the Design of Digital Recording Devices” (2008). Dib’s installations and compositions range from the ethnographic to the experimental, and have been shown in Montreal, New York, New Orleans, Paris and Houston.

Tyr Umbach

Tyr Umbach is a scientist and student of mathematics, fine arts and philosophy. Since 2009, Tyr has been a contributor to the realtime video and state engine components of the TML’s Ozone media choreography platform, and the Il Y A video membrane installation, with much of his current TML efforts concentrated upon dynamical systems and continuous state machines. His fields of interest include algebra, topology, and geometry, and their applications to physics, chemistry, natural science, and philosophy.

Magdalena Isabela Victoria Olszanowski

Concordia University. She obtained her BA honours summa cum laude from the University of Toronto and an MA in Communication and Culture from York University. She has studied at the University of Amsterdam on full scholarship. A certificate in Creative Writing from the University of Toronto school of Continuing Education also hangs on her mom’s wall.Born in Warsaw, Poland, she is a sujet-en-proces, existing within the contours of relations. Her work grounds itself in the multi-faceted topology of surveillance and process/movement. Magdalena’s research topics include the body as an ‘open system’ prepared for ‘abstract sex’, granular synthesis, process philosophy’s potential in re-framing mapping practices, melancholia, emerging media and the liminal spaces of the ‘not yet’ in immersive environments. By the by, she identifies as an arts-based researcher with lax hygiene and no social graces.

Michael Fortin

Michael has expertise and interest in computer graphics, artificial intelligence and
artificial life. He has developed some core optimizations of the physics and time-based
video computation, as well as GL projects. He is working on some applications of
machine vision as part of the Ozone media choreography team. He received is MA in
Computer Science at Concordia University.

Navid Navab

Navid Navab | | is a Montreal based composer, media alchemist, audio-visual sculptor, perSonifier, gestureBender, phono-menologist, and all around multidisciplinary artist.

Interested in the poetics of schizophonia, materiality, and embodiment, his work investigates the transmutation of matter and the enrichment of its inherent performative qualities. Making the imperceptible palpable, Navid uses gestures, rhythms and vibration from everyday life as basis for real-time compositions, resulting in augmented acoustical-poetry and painterly light that enchants improvisational and pedestrian movements. A graduate of Ontario Royal Conservatory of Music, Concordia University’s Electroacoustics and Computational Arts program, and McGill Music Technology, Navid has been leading experiments for the past several years creating deeply expressive intermedia instruments synthesizing his research at IRCAM, CRIMMT, CNMAT, Topological Media Lab and Matralab, while using phenomenological studies to inform the creation of computationally-augmented performance environments.

His pieces, which which take on the form of gestural sound compositions, responsive architecture, site specific interventions, theatrical interactive installations, kinetic sound sculptures and multimodal improv-based performances, have been presented internationally at diverse venues including Canadian Center for Architecture, Festival du Nouveau Cinema, eArts Shanghai, MUMUTH Austria, Roulette New York, Western Front Vancouver, McCord Museum, Musée d’art Contemporain de Montréal, Timisoara Romania, Contemporary Arts Museum Houston, International Digital Arts Biennial, Musiikin Aika Finland, HKW Berlin, Festival International Montréal/Nouvelles Musiques, Suoni Per il Popolo Festival, CURRENTS Santa Fe,  CCRMA Palo Alto, CINARS Biennale, Electric Eclectics Meaford, and TANGENTE mtl among others.

His currently ongoing decade long series such as Practices of Everyday Life, and GestureBending interventions reflect Navid’s continued interest in the enactment of dynamic performative ecologies that allow for poetic and virtuosic improvisation with computationally enriched materials and environments. 

“Underlying this array imaginative creations is a playful awareness of the relationship between gesture, materiality, and sound. By enchanting the inherent properties of materials we know from everyday life with potential for play, Navid reacquaints us with the magic and wonder of our embodied experience.”   

-Rachel Elliott, international institute for critical studies in improvisation