Deep below Morrison Hall, quadraphonic artist Suzanne Ciani sat in a soundproof recording studio reciting strings of non sequiturs with varying intonations to train a deepfake program.
Researchers and engineers used Ciani’s non sequiturs to generate training data for a speech synthesizer program, or a deepfake, that will pre-record messages for Ciani’s fans to buy on vinyl records.
“The lazy cow lays in the cool grass,” Ciani said in an excited voice during the recording. “Hop over the fence and plunge in. The frosty air passed through the coat. Adding fast leads to wrong sums.”
Justin Norman, a doctoral student at UC Berkeley’s School for Information, said Ciani’s speech synthesizer will be able to generate messages never recorded before. Other programs including Apple’s Siri use artificial intelligence, or AI, that samples information from a certain voice.
Norman added that the speech synthesizer will have attributes that make it seem more realistic including emotion, intonation, continuous flow and normal speech cadence.
Rafael Valle, who has an interdisciplinary doctorate from UC Berkeley, recorded Ciani on Friday. He said the synthesizer will also have the ability to modulate accent, tone and pitch.
Valle added that the synthesizer can potentially recite messages in any language, given sufficient sophistication.
“Read all the questions in excitement or happily – the synthesizer has the ability to read the emotion, not the content,” Valle told Ciani during the recording session. “You’re laughing already, that’s great. That brings us as close as possible to a real voice. Laugh as much as you want.”
Project producer KamranV noted that the “genuine” messages from Ciani, which she never actually said, are an homage to Don Buchla.
Buchla, who passed away in 2016, invented an electronic musical instrument called the modular synthesizer, studied physics on campus and employed Ciani after she received her master’s degree in composition from UC Berkeley.
“It’s like an autograph,” Ciani said. “It’s a personalization … You have the message and the receiver of the message and the content is made in that exchange – they are creating the message when they receive it.”
Ciani added that she would never allow anyone to use the speech synthesizer program to generate messages she did not approve of beforehand. The purpose of the project is not to generate disinformation, she added, but rather to be playful with the novel technology and communicate directly in an indirect way.
Recent uses of deepfake technology make Ciani “nervous.” She said she feels “threatened” that the line between real and automated has become much blurrier. “Alternate realities,” such as movies, already exist, she noted, but the difference between movies and some deepfake technology is that people know movies are fake.
“These things are starting to invade our real life and people are having a hard time dealing with this,” Ciani said. “We have to be able to bring doubt or skepticism, analysis, assessment of where they come from.”
Contact Eric Rogers at erogers@dailycal.org, and follow him on Twitter at @eric_rogers_dc.