Capturing the magic of musicians in motion
Artist duo Cleary Connolly use motion capture technology on members of Cork Youth Orchestra to bring a "musical magic lantern" that joins the dots between art and science to Fitzgerald's Park.
In the early 1970s, Swedish psychologist Dr Gunnar Johansson began experimenting with the way in which we perceive motion.
He fascinated the scientific community with his demonstration that the human brain, given just a few highlighted points on a body in motion, will rapidly join the dots and identify complete figures and their actions.
“He showed that 13 moving points could really describe a body very fully,” artist Anne Cleary tells me. “You could tell the weight of the person, the age of the person, the gender as well because women move differently to men. But only when they’re moving. If they’re not, all you’ll see are dots.”
For example, the below image may look like an unfamiliar constellation of stars or a random bunch of dots:
But it’s actually a still from a short film demonstrating Johansson’s discoveries: if you watch the video on Youtube (from about six minutes onwards), you will very clearly perceive it as a couple walking along holding hands.
The concept is called Biological Motion Perception and, as well as giving rise to many technologies such as those used in CGI for movies, it has fascinated Anne Cleary and her partner and fellow artist Denis Connolly since they first stumbled upon it in 2009.
The fact that we are biologically hard-wired to reconstruct form from movement is probably a lot to do with our hunter-gatherer past, Anne explains.
“When you’ve got this small amount of dots, the human mind reconstructs the body,” she says. “The reason we can do that is because thousands of years ago, if you were hunting in a forest, you saw a movement through the leaves of maybe a lion or another person about to attack you, you needed to respond quickly.”
“So we have the facility to construct biological motion from very tiny fragments of movement. It’s a really essential human characteristic, this ability we have to construct from very small amounts of visual information.”
Anne and Denis have worked together as Cleary Connolly for years: as well as an art practice partnership they are life partners and have lived in Paris since the nineties, where they have raised twin daughters.
As artists, they are known for their integrations of science and art and have frequently worked with visual and perceptual phenomena: as The School of Looking they have built wearable YellowBlue helmets for Talking Heads singer David Byrne’s theatrical show Theater of the Mind, and have embarked on innumerable workshops, installations and exhibitions, including Invisible Light at the Crawford Art Gallery in an ill-fated time window during 2020 that saw them have to move much of its planned educational programme online as the gallery closed repeatedly due to Covid restrictions.
Last year, they launched their Eco Showboat project and travelled the Shannon and Erne waterways on a small solar-powered yacht, the Mayfly, working with artists and scientists to explore and highlight the problems of climate change.
And they’ve been using Johansson’s discovery for many years in their work; they had worked with dancers and athletes, but it was when they put famous conductor Sir Simon Rattle in a Motion Capture (MoCap) suit for a piece for London’s Barbican Centre in 2017 that it they realised how powerful it was to connect biological motion perception and music.
“We were convinced by the beauty of looking at moving dots and seeing bodies in it, and for us that’s the beauty of art,” Denis Connolly says. “But for the public it was hard to get into it, so I think the music connection gave it an extra dimension. A conductor is always silent, but a conductor’s movements induce music.”
Lucky 13
Cleary Connolly called this piece Less than thirteen because they were dispensing with Johansson’s 13 points and playing with the bare minimum number of points on a body it takes for a viewer to start perceiving a human in motion.
“We realised that’s when it becomes exciting: when you’re not giving enough information and when the brain works its magic anyway,” Denis says.
The effect is extraordinary: with sometimes as few as five illuminated points on Sir Simon Rattle’s arms and hands as he’s conducting, the brain clearly and quickly deciphers a human form.
Although the piece was silent, some viewers were even able to figure out that it was Elgar’s Enigma Variations that Sir Rattle was conducting for the piece.
Capturing the motion of Cork Youth Orchestra
Now, Cleary Connolly are returning to Cork where, during December and January, their piece The Music Box will be on display at the Lord Mayor’s Pavilion in Fitzgerald’s Park.
Using a Neuron3 Motion Capture suit and working with the Siciliana Quartet from Cork Youth Orchestra (CYO) and their director Tomás McCarthy, the artists have created an installation that they describe as a "musical magic lantern" that will bring the young musicians’ movements to life as moving dots.
The MoCap suit creates an avatar from the captured movement and is technology most frequently used for film, Denis explains: “Normally, it’s used for animation as a cinema tool. So if you want your animated Spiderman to make realistic while he’s punching Doctor Octopus, you get an actor to do it and then transfer that to the animated Spiderman.”
The four young musicians of CYO recorded the fourth movement of The American Quartet by Czech composer Antonin Dvořák.
The choice of music was left in the hands of the musicians and Tomás McCarthy. But Denis says it’s a fitting one.
“All I said to Tomás was that the Pavilion was built at the beginning of the 20th Century for Cork International Exhibition, so maybe we need to keep that sort of Belle Epoque sort of feel,” he says.
Tomás later tells me on the phone that the musicians both liked and were already familiar with the music. And that both he and the musicians are very excited about the launch, and about seeing the project’s outcome.
Anne describes The Music Box as one of Cleary Connolly’s “most challenging pieces ever,” largely for production reasons rather than anything to do with the art itself.
Working with Cork’s Sample-Studios, the artists found the piece, which has now been three years in the making, first postponed due to Covid, followed by a series of other stumbling blocks: a world-wide shortage of Titanium meant that MoCap suits were in low supply, and even Brexit popped its head up when the artists found a suit for loan in London that didn’t materialise due to concerns about transporting it internationally.
There’s even been the small matter of convalescence from a major surgery for Anne along the way.
But in November, Denis came to Cork to finally capture the musicians playing. They wore headphones and played in time to their recording so that the tempo of their movements would be perfectly synced to the recording.
“For them, the fascination was putting on the suit and the gloves with all the sensors on them, and then seeing the avatar,” Denis says. “There was a real comical moment at the end where they started taking off the suit and we left the avatar on the screen: the computer programme continues to try to make sense of the figure while you’re taking off the sensors so it starts doubling over or going flat on the floor with the arms in the air.”
Four projections will be visible from the outside of the Lord Mayor’s Pavilion, and visitors will hear the music too.
As with Less than thirteen, it’s the minimalism of The Music Box that lets the viewer marvel at the power of their own mind to perceive human motion.
“The thing we had to resist was the temptation to elaborate, especially for the CGI artist we were working with, who wanted to make them into fancy parallelograms or balls of string,” Denis says.
“We’re relying on the spectator to let themselves go and extrapolate forms. You’re paring everything down to these dots, and yet out of this absolute austere minimalism your mind flows with human bodies, and with music.”
Cleary Connolly’s The Music Box, will be visible at The Lord Mayor's Pavilion, Fitzgerald’s Park, Cork from December 9 to January 21, 2023. The launch is at 4pm on December 9.