George Tzanetakis

George Tzanetakis

Rise of the robot musicians

UVic professor George Tzanetakis on track to create computer music systems that can jam with humans

The thing about robot musicians is they’ll never trash a hotel room, show up to a gig drunk or break up a band after meeting an eccentric performance artist.

At least not yet.

The music-computer lab of George Tzanetakis at the University of Victoria is ground zero for the rise of robot musicians. But don’t think of it like anthropomorphic C-3PO strumming a mandolin – it’s more like a series of mallets linked to solenoids on a drum, backed by sophisticated software. Tzanetakis effectively wants to teach music to a computer, and for the computer to pick up on musical cues while jamming with humans.

“When you play sound, a musician hears what is happening. We are trying to add the ability to understand music to an artificial agent that performs,” says Tzanetakis, an assistant professor in the department of computer science, and in the department electrical and computer engineering – and the school of music.

“The idea is to make the system musically intelligent, to have robotic musicianship.”

It remains early days for robot jazz bands. Mechanical musicians are largely limited to percussion instruments due to increasing engineering complexity as each mallet and its actuator is added to the mix, and to the software that crunches the music in real time to adjust the tempo and volume.

“We want the musicality of the system to learn how to slow down, speed up, understand if it is too loud. We are still working on making sure it doesn’t get drunk,” Tzanetakis jokes.

Of course computers, even advanced learning systems, don’t understand music or sound or much of anything else – they just crunch data in sophsticated ways. Gabrielle Odowichuk, who worked in UVic computer-music lab as a masters student, said completely computer-driven music seems unlikely. Computers still can’t connect on an emotional level.

“You need a human component or its boring. All robots need a human component,” says Odowichuk who worked on gestural control of sound. “Often you can program in more randomness to make it more human. But when it comes to conveying something and connecting with an audience, computers lack expressiveness.”

Tzanetakis sees robots and computer musicality as the natural evolution in how technology influences how music is made. To him, it’s no different than the person who first strung wires and called it a piano, or plugged in a wah-wah pedal.

“Using computers and robots is the same process,” he says. “Musicians accept synthesizers, drum machines and DJ sets. If anything a robot is more innocent than a drum machine. It’s just another technical dimension.”

How this work will eventually influence the broader music world is hard to predict. Tzanetakis, a 37-year-old native of Greece who earned his PhD in computer science from Princeton University in 2002, tends to be about five years ahead of current popular technology.

Tzanetakis’ early work on acoustic signal processing laid the groundwork for popular apps such as Shazam, which can recognize songs by holding a smartphone up to a speaker. He’s helped develop melody-matching systems that identify songs by users humming or singing. His music sensing algorithms are at the heart Smule’s AutoRap app, which creates a rap song out of any set of sounds, and the Ocarina app, which transforms an iPhone into a flute.

Of course, when he began working on what’s called audio fingerprinting – systems that analyze, dissect and find patterns in acoustic signals – smartphone technology didn’t exist.

“If I do my job right, stuff I’m doing now will be commercial in about five years,” he says. “What is commercial now is stuff myself and others were doing 10 years ago. Ten years ago there was no iPod, no iTunes. The world was a different place.”

Tzanetakis expects the next big creative leaps to involve computer systems that can analyze and extract individual instruments and voices from complex music scores. The human ear can pick our individual conversations in a noisy party – computers cannot.

“It’s really a hard problem, far from being solved, but is actively researched,” he said.

Despite being at the forefront of intertwining computers and music, Tzanetakis, a jazz musician and pianist before he became a computer scientist and engineer, admits he doesn’t always practice what he preaches.

“I’m a sax player. I have typically avoided playing with technology.”

editor@saanichnews.com

 

 

Get local stories you won't find anywhere else right to your inbox.
Sign up here

Just Posted

Thousands protested in Victoria following the death of George Floyd in the U.S. (Black Press Media file photo)
B.C. is not exempt: New report documents 150 years of racism and the fight against it

Booklet marks province’s 150th anniversary with call for transparency, change

Aerial view of the Capital Regional District residuals treatment facility at Hartland Landfill where residual solids are turned into Class A biosolids. (Photo courtesy CRD)
Plant closure sends more biosolids to Hartland Landfill

Saanich residents are concerned they were never consulted

Sgt. Sandrine Perry in the Oak Bay Police Department interview room that has been softened with household features to better accommodate survivors of domestic abuse and sexual assault. (Travis Paterson/News Staff)
Oak Bay police interview room gets a makeover

Room made less daunting for victims of trauma

BC Housing ensures that by March 31, shelter will be available to all people living outside. (Black Press Media file photo)
All unhoused Victoria residents will be offered shelter by March 31, says BC Housing

BC Housing working to secure shelter locations in coming weeks

Robert Schram, here seen in January 2016, died Saturday, according to a friend. (Black Press Media file photo)
Sidney, Saanich Peninsula mourn the death of Mr. Beads

Bead artist Robert Schram was a familiar, well-loved figure in Sidney and beyond

Langley resident Carrie MacKay shared a video showing how stairs are a challenge after spending weeks in hospital battling COVID-19 (Special to Langley Advance Times)
VIDEO: Stairs a challenge for B.C. woman who chronicled COVID-19 battle

‘I can now walk for six (to) 10 minutes a day’

The Nanaimo Clippers in action at Frank Crane Arena in early 2020. (News Bulletin file photo)
Nanaimo Clippers for sale, owner says hockey won’t be back to normal any time soon

Wes Mussio says he’s had numerous inquiries about the junior A club already

Provincial health officer Dr. Bonnie Henry updates B.C.’s coronavirus situation, May 8, 2020. (B.C. government photo)
B.C.’s weekend COVID-19 cases: 532 Saturday, 508 Sunday, 438 Monday

Fraser Health still has most, eight more coronavirus deaths

B.C. Attorney General David Eby speaks in the legislature, Dec. 7, 2020. Eby was given responsibility for housing after the October 2020 provincial election. (Hansard TV)
B.C. extends COVID-19 rent freeze again, to the end of 2021

‘Renoviction’ rules tightened, rent capped to inflation in 2022

Face mask hangs from a rear-view mirror. (Black Press image)
B.C. CDC unveils guide on how to carpool during the pandemic

Wearing masks, keeping windows open key to slowing the spread of COVID-19

Churches, including Langley’s Riverside Calvary Church, are challenging the regulations barring them from holding in-person worship services during COVID-19. (Langley Advance Times file)
Det. Sgt. Jim Callender. (Hamilton Police Service screenshot)
B.C. man dead, woman seriously injured after shooting in Hamilton, Ont.

The man was in the process of moving to the greater Toronto area, police say

Police have identified the vehicle involved in the Feb. 14 hit-and-run in Chemainus and are continuing to investigate. (Black Press Media files)
Police seize and identify suspect vehicle in hit-and-run

Investigation into death expected to be lengthy and involved

(Black Press file photo)
Child in critical condition, homicide investigators probe incident near Agassiz

The child was transported to hospital but is not expected to survive

Most Read