George Tzanetakis

George Tzanetakis

Rise of the robot musicians

UVic professor George Tzanetakis on track to create computer music systems that can jam with humans

The thing about robot musicians is they’ll never trash a hotel room, show up to a gig drunk or break up a band after meeting an eccentric performance artist.

At least not yet.

The music-computer lab of George Tzanetakis at the University of Victoria is ground zero for the rise of robot musicians. But don’t think of it like anthropomorphic C-3PO strumming a mandolin – it’s more like a series of mallets linked to solenoids on a drum, backed by sophisticated software. Tzanetakis effectively wants to teach music to a computer, and for the computer to pick up on musical cues while jamming with humans.

“When you play sound, a musician hears what is happening. We are trying to add the ability to understand music to an artificial agent that performs,” says Tzanetakis, an assistant professor in the department of computer science, and in the department electrical and computer engineering – and the school of music.

“The idea is to make the system musically intelligent, to have robotic musicianship.”

It remains early days for robot jazz bands. Mechanical musicians are largely limited to percussion instruments due to increasing engineering complexity as each mallet and its actuator is added to the mix, and to the software that crunches the music in real time to adjust the tempo and volume.

“We want the musicality of the system to learn how to slow down, speed up, understand if it is too loud. We are still working on making sure it doesn’t get drunk,” Tzanetakis jokes.

Of course computers, even advanced learning systems, don’t understand music or sound or much of anything else – they just crunch data in sophsticated ways. Gabrielle Odowichuk, who worked in UVic computer-music lab as a masters student, said completely computer-driven music seems unlikely. Computers still can’t connect on an emotional level.

“You need a human component or its boring. All robots need a human component,” says Odowichuk who worked on gestural control of sound. “Often you can program in more randomness to make it more human. But when it comes to conveying something and connecting with an audience, computers lack expressiveness.”

Tzanetakis sees robots and computer musicality as the natural evolution in how technology influences how music is made. To him, it’s no different than the person who first strung wires and called it a piano, or plugged in a wah-wah pedal.

“Using computers and robots is the same process,” he says. “Musicians accept synthesizers, drum machines and DJ sets. If anything a robot is more innocent than a drum machine. It’s just another technical dimension.”

How this work will eventually influence the broader music world is hard to predict. Tzanetakis, a 37-year-old native of Greece who earned his PhD in computer science from Princeton University in 2002, tends to be about five years ahead of current popular technology.

Tzanetakis’ early work on acoustic signal processing laid the groundwork for popular apps such as Shazam, which can recognize songs by holding a smartphone up to a speaker. He’s helped develop melody-matching systems that identify songs by users humming or singing. His music sensing algorithms are at the heart Smule’s AutoRap app, which creates a rap song out of any set of sounds, and the Ocarina app, which transforms an iPhone into a flute.

Of course, when he began working on what’s called audio fingerprinting – systems that analyze, dissect and find patterns in acoustic signals – smartphone technology didn’t exist.

“If I do my job right, stuff I’m doing now will be commercial in about five years,” he says. “What is commercial now is stuff myself and others were doing 10 years ago. Ten years ago there was no iPod, no iTunes. The world was a different place.”

Tzanetakis expects the next big creative leaps to involve computer systems that can analyze and extract individual instruments and voices from complex music scores. The human ear can pick our individual conversations in a noisy party – computers cannot.

“It’s really a hard problem, far from being solved, but is actively researched,” he said.

Despite being at the forefront of intertwining computers and music, Tzanetakis, a jazz musician and pianist before he became a computer scientist and engineer, admits he doesn’t always practice what he preaches.

“I’m a sax player. I have typically avoided playing with technology.”

editor@saanichnews.com

 

 

Get local stories you won't find anywhere else right to your inbox.
Sign up here

Just Posted

A Nexii roof panel is lifted during construction of a Starbucks in Abbotsford. Alexzi Building Solutions will be building a manufacturing plant in Langford or North Cowichan to produce the sustainable construction panels. (Photo courtesy of Alexzi Building Solutions)
Langford eyed for facility to make green building alternative to concrete

Langford, North Cowichan possible sites for plant to create sustainable construction panels

Local MP Elizabeth May says the public has a right to know the identity of the company that plans to operate the massive warehouse proposed for Sidney on airport lands but residents who want to stop the project would probably have to go through the courts. (Black Press Media File)
MP Elizabeth May says public has right to know identity of Sidney warehouse operator

Residents wanting to stop the project would probably have to go through the courts, said May

Bukwila by Art Thompson, set at the steps of Lansdowne campus’ Wilna Thomas Cultural Centre, has been welcoming students to campus since 1997. (Photo courtesy of Camosun College)
World-renowned artist’s legacy lives on at Camosun College

Art Thompson made untold contributions toward Indigenous education, art and advocacy

Did you know, according to the CRD, every person produces an average of 185–200 litres of wastewater per day? Here’s where most of it gets treated, at the new wastewater treatment facility at McLoughlin Point in Esquimalt. (CRD image)
View Royal signs on to wastewater funding plan

Capital Regional District requesting to borrow up to $34.3 million to upgrade infrastructure

(The Canadian Press)
Trudeau won’t say whether Canada supports patent waiver for COVID-19 vaccines

‘Canada is at the table to help find a solution’

A map showing where the most number of cases were recorded from April 23 to 29. This map, revealing a breakdown of infections by neighborhood, was pulled from a data package leaked to the Vancouver Sun last week (and independently verified).
36 Abbotsford schools flagged for COVID-19 exposures in the last 2 weeks, shattering record

Clearbrook Elementary recorded an ‘exposure’ on all 11 school days

Island Health has confirmed COVID-19 exposures at Ecole des Deux Mondes in Campbell River on May 4 and 5, and at Mill Bay Nature School in Mill Bay on April 28, 29, 30 and May 3. (Metro Creative photo)
Two new COVID-19 school exposures confirmed by Island Health

Health authority contacting anyone exposed at Ecole des Deux Mondes, Mill Bay Nature School

Canada’s chief public health officer is reminding Canadians even those who are fully vaccinated are not immune from transmitting the COVID-19. THE CANADIAN PRESS/Adrian Wyld
Canada’s top doctor warns full vaccination does not equal full protection from COVID-19

Post-inoculation, Theresa Tam says the risk of asymptomatic infection and transmission is far lower but not obsolete

The dash cam footage, taken May 7 at 8:18 a.m. belonged to the driver of a southbound vehicle that recently travelled out of the tunnel. (Reddit/Screen grab)
VIDEO: Dash cam captures dramatic rollover crash on Highway 99

Only one person sustained injuries from the collision, says B.C. Ambulance Services

Chevy stranded on a ledge above a rocky canyon at Mimi Falls near Logan Lake, April 28, 2021. (Photo credit: Margot Wikjord)
Police officer and fire chief team up in risky rescue of stranded dog near Logan Lake

Chevy, a rescue dog, needed rescuing again after getting stuck on a ledge above rocky canyon

Police were on the scene of a fatal shooting in Abbotsford. (Black Press Media files)
B.C. government to give more than $8 million for programs to curb gang violence

221 not-for-profit projects led by local governments and school districts among others will receive a one-time grant

Gord Judson steers his log truck down a forest service road, using two-way radio and call signals to mark his position for oncoming traffic. (B.C. Forest Safety Council)
Planning some B.C. wilderness fishing? Don’t catch a log truck

Remote recreation areas bracing for heavy pandemic pressure

Most Read