Two UT students have made translating thoughts into a display of lights, sounds and 3-D graphics possible in their latest artistic exhibition.
Yago de Quay and Joao Beira, both working toward dual doctoral degrees in digital media, have developed a rare artistic performance that intersects the arts with new technologies. They have created “BioMediation,” a choreographed display of 3-D graphics and sounds that incorporates the performer’s thoughts.
“We’re trying to see how we can start using our brains as machines that produce the art itself, without the body,” de Quay said.
De Quay and Beira are both part of the UT Portugal program. The program works to advance exploration of emerging technologies across the nation of Portugal by offering extensive studies in digital media. It requires de Quay and Beira to complete part of their coursework in Portugal and part of it in Austin.
In mid-January, de Quay and Beira began work on “BioMediation.” Their collaborative work in interactive digital art performances in the past led to their recent commission by the Ammeman Center for Arts and Technology at Connecticut College. This commission gave them the funding and resources needed to complete their research for “BioMediation.”
“This technology lets you actually use your brain and thoughts as an interface for creative content to establish a way to express sounds and visualizations,” Beira said.
The project is a complicated system that integrates the mind of the user with a depth-sensor camera that uses technology similar to an Xbox Kinect. It also uses a computer and an electroencephalogram headset, which tracks electric activity on the scalp.
First, the headset is attached to a person’s head. The way the person feels and thinks creates the electric activity that is then mapped and translated by the computer into sound and visual compositions in real time. In addition, the camera tracks physical movement and spatial positioning, then projects the visuals and sound back onto the user in a virtual reality display.
“A good analogy is we are using brainwaves as sound waves and imagination as visuals,” Beira said. “It’s all very abstract and experimental.”
De Quay and Beira recently took “BioMediation” to Connecticut to show the progress they have made. Once the headset is attached to de Quay, he positions himself in front of the camera and meditates. His thoughts as he meditates are recorded and translated through the system and displayed back onto him in an array of graphics and sounds.
“The idea is that I will be creating this environment by thinking about it and creating these images and sounds that are being shot at me with a projector with speakers,” de Quay said.
De Quay and Beira hope to perform “BioMediation” at a School of Information showcase in April. They will have an installation that allows audiences to experience their project firsthand. “For example, by going into a meditative state and closing their eyes slowly and deeply, they can alter the sound from a busy, noisy area to a nature inspired sound,” Beira said. “And the visuals will also change.”
UT music professor Bruce Pennycook, who specializes in new media, audio technologies and interactive music performances, supervises de Quay and Beira’s doctoral studies and projects.
Pennycook said people like him, de Quay and Beira have been working on this type of artistic expression for a decade. With technology advancing more and more rapidly, he said he thinks highly of what the two students have accomplished during their time in Austin.
“To put things in the simplest of terms, they’re working to make art,” Pennycook said.