The neuroscience of Bob Dylan's genius  ...  hmm

How do we have insights, and where does inspiration come from? Jonah Lehrer goes inside Bob Dylan's brain to find out

'What Dylan felt was the itch of an imminent insight, the tickle of lyrics that needed to be written down.' Illustration: Ray Smith for the Guardian

Bob Dylan looks bored. It's May 1965 and he's slumped in an armchair at the Savoy hotel in London. The camera filming the documentary Don't Look Back, about Dylan's tour, turns away – Dylan's weariness feels like an accusation – and starts to pan around the room, capturing the ragged entourage of folkies and groupies following the singer on the final week of his European tour.

For the previous four months, Dylan had been struggling to maintain a gruelling performance schedule. He'd been paraded in front of the press and asked an endless series of inane questions. At times, Dylan lost his temper and became obstinate with reporters. "I've got nothing to say about these things I write," he insisted. "There's no great message. Stop asking me to explain."
Although Dylan's creativity remained a constant – he wrote because he didn't know what else to do – there were increasing signs that he was losing interest in creating music. The only talent he cared about was being ruined by fame. The breaking point probably came after a brief vacation in Portugal, where Dylan got a vicious case of food poisoning. The illness forced him to stay in bed for a week, giving the singer a rare chance to reflect. "I realised I was very drained," Dylan would later confess. "I was playing a lot of songs I didn't want to play. It's very tiring having other people tell you how much they dig you if you yourself don't dig you."
In London at a sold-out Royal Albert Hall, Dylan told his manager he was quitting the music business. He was finished with singing and songwriting and was going to move to a tiny cabin in Woodstock, New York.
Every creative journey begins with a problem. It starts with a feeling of frustration, the dull ache of not being able to find the answer. When we tell one another stories about creativity, we tend to leave out this phase of the creative process. We neglect to mention those days when we wanted to quit, when we believed that our problems were impossible to solve. Instead, we skip straight to the breakthroughs. The danger of telling this narrative is that the feeling of frustration – the act of being stumped – is an essential part of the creative process. Before we can find the answer – before we probably even know the question – we must be immersed in disappointment, convinced that a solution is beyond our reach. It's often only at this point, after we've stopped searching for the answer, that the answer arrives. All of a sudden, the answer to the problem that seemed so daunting becomes incredibly obvious.

This is the clichéd moment of insight that people know so well from stories of Archimedes in the bathtub and Isaac Newton under the apple tree. When people think about creative breakthroughs, they tend to imagine them as incandescent flashes, like a light bulb going on inside the brain.

These tales of insight all share a few essential features that scientists use to define the "insight experience". The first stage is the impasse. If we're lucky, however, that hopelessness eventually gives way to a revelation. This is another essential feature of moments of insight: the feeling of certainty that accompanies the new idea. After Archimedes had his eureka moment – he realised that the displacement of water could be used to measure the volume of objects – he immediately leaped out of the bath and ran to tell the king about his solution. He arrived at the palace stark naked and dripping wet.

At first glance, the moment of insight can seem like an impenetrable enigma. We are stuck and then we're not, and we have no idea what happened in between. It's as if the cortex is sharing one of its secrets. The question, of course, is how these insights happen.

It was the early 1990s and Mark Beeman, a young scientist at the National Institutes of Health in America, was studying patients who had suffered damage to the right hemisphere of the brain. "The doctors would always tell these people, 'You're so lucky,'" Beeman remembers. "They'd go on about how the right hemisphere was the minor hemisphere – it doesn't do much, and it doesn't do anything with language." Those consoling words reflected the scientific consensus that the right half of the brain was mostly unnecessary. In his 1981 Nobel lecture, the neuroscientist Roger Sperry summarised the prevailing view of the right hemisphere at the time: the right hemisphere was "not only mute and agraphic but also dyslexic, word-deaf and apraxic, and lacking generally in higher cognitive function". In other words, it was thought to be a useless chunk of tissue.

But Beeman noticed that many patients with right hemisphere damage nonetheless had serious cognitive problems even though the left hemisphere had been spared. "Some of these patients couldn't understand jokes or sarcasm or metaphors," Beeman says. "Others had a tough time using a map or making sense of paintings."
At first Beeman couldn't figure out what all these deficits had in common. What did humour have to do with navigation? And then, just when Beeman was about to give up, he had an idea – perhaps the purpose of the right hemisphere was doing the very thing he was trying to do: find the subtle connections between seemingly unrelated things. Beeman realised that all of the problems experienced by his patients involved making sense of the whole, seeing not just the parts but how they hang together.
Take the language deficits caused by right hemisphere damage. Beeman speculated that, while the left hemisphere handles denotation – it stores the literal meanings of words – the right hemisphere deals with connotation, or all the meanings that can't be looked up in the dictionary. Metaphors are a perfect example of this. From the perspective of the brain, a metaphor is a bridge between two ideas that, at least on the surface, are not equivalent or related. When Romeo declares that "Juliet is the sun", we know that he isn't saying his beloved is a massive, flaming ball of hydrogen. We understand that Romeo is trafficking in metaphor. She might not be a star, but perhaps she lights up his world in the same way the sun illuminates the Earth.
How does the brain understand the line "Juliet is the sun"? The left hemisphere focuses on the literal definition of the words, but that isn't particularly helpful. We can grasp the connection between the two nouns only by relying on their overlapping associations, by detecting the nuanced qualities they might have in common. This understanding is most likely to occur in the right hemisphere, since it's uniquely able to zoom out and parse the sentence from a more distant point of view.
It doesn't apply just to language. A study conducted in the 1940s asked people with various kinds of brain damage to copy a picture of a house. Patients reliant on the left hemisphere because the right hemisphere had been incapacitated depicted a house that was clearly nonsensical: front doors floated in space; roofs were upside down. However, they carefully sketched its specifics and devoted lots of effort to capturing the shape of the bricks in the chimney or the wrinkles in the window curtains. In contrast, patients who were forced to rely on the right hemisphere tended to focus on the overall shape of the structure. Their pictures lacked details.

In 1993, Beeman heard a talk on moments of insight by Jonathan Schooler, a psychologist now at the University of California at Santa Barbara. Schooler presented the results of a simple experiment: he'd put undergraduates in a room and given them a series of difficult creative puzzles. Here's a sample question: a giant inverted steel pyramid is perfectly balanced on its point. Any movement of the pyramid will cause it to topple over. Underneath the pyramid is a $100 bill. How do you remove the bill without disturbing the pyramid?

Almost everyone begins by visualising the pyramid. Your next thought probably involves some sort of crane that would lift the pyramid. (Alas, such a contraption violates the rules.) Then you might imagine a way of sliding the money out without tearing the bill. Unfortunately, for most people, no workable solutions come to mind, which is why they reach the impasse stage. The subject gets flustered and frustrated. At this point, Schooler began giving the subjects hints. He subliminally flashed them a sentence with the word fire or told them to think about the meaning of remove. "We'd give people these funny goggles that allowed us to flash hints to one eye at a time," Schooler says. "And it was startling how you could flash a really obvious hint to the right eye [and hence left hemisphere] and it wouldn't make a difference. They still wouldn't get it. But then you'd flash the exact same hint to the other eye, and it would generate the insight. (If you're still wondering, the solution is to set the hundred-dollar bill on fire.)
While the left hemisphere was trying to lift the pyramid into the air – the obvious way to "remove" the money – the right hemisphere was thinking about alternative approaches. "I realised that moments of insight could be a really interesting way to look at all these skills the right hemisphere excelled at," Beeman says.
In order to isolate the brain activity that defined the insight process, Beeman needed to compare moments of insight to answers that arrived by conscious analysis, that is, by people methodically testing ideas one at a time. In conscious analysis, people have a sense of their progress and can accurately explain their thought processes. The problem is solved through diligence and hard work; when the answer arrives, there is nothing sudden about it. Unfortunately, all of the puzzles used by scientists to study insight required insight. Either they were solved in a sudden "Aha!" moment or they weren't solved at all. This was Beeman's challenge: to come up with a set of puzzles that were often solved by insight, but not always. In theory, this would allow him to isolate the unique neural patterns that defined the insight process.
The puzzles go like this: a subject is given three different words, such as age, mile and sand, and asked to think of a single word that can form a compound word or phrase with each of the three. (In this case, the answer is stone: stone age, milestone, sandstone.) The subject has 15 seconds to solve the question before a new puzzle appears. If he comes up with an answer, he presses the space bar on the keyboard and says whether the answer arrived via insight or analysis.

Beeman teamed up with John Kounios, a psychologist at Drexel University. The first thing they discovered was that, although it seemed that the insight answer appeared out of nowhere, the brain had been laying the groundwork for the breakthrough. The process began with an intense mental search as the left hemisphere started looking for answers in all the obvious places. Because Beeman and Kounios were giving people word puzzles, they saw additional activation in brain areas related to speech and language. This left-brain thought process, however, quickly got tiring – it took only a few seconds before the subject said he'd reached an impasse and couldn't think of the right word. But these negative feelings are actually an essential part of the process because they signal that it's time to try a new search strategy. Instead of relying on the literal associations of the left hemisphere, the brain needs to shift activity to the other side, to explore a more unexpected set of associations.

What's surprising is that this mental shift often works. Because we feel frustrated, we start to look at problems from a new perspective. "You'll see people bolt up in their chair and their eyes go all wide," says Ezra Wegbreit, a graduate student in the Beeman lab. The suddenness of the insight is preceded by an equally sudden burst of brain activity. Thirty milliseconds before the answer erupts into consciousness, there's a spike of gamma-wave rhythm, which is the highest electrical frequency generated by the brain. Gamma rhythm is believed to come from the binding of neurons: cells distributed across the cortex draw themselves together into a new network that is then able to enter consciousness.
Where does this burst of gamma waves come from? Beeman and Kounios went back, analysed the data and discovered the "neural correlate of insight": the anterior superior temporal gyrus (aSTG). This small fold of tissue, located on the surface of the right hemisphere just above the ear, became unusually active in the seconds before the epiphany. (It remained silent when people solved the word puzzles by analysis.) The activation of the cortical circuit was sudden and intense, a surge of electricity leading to a rush of blood.
Beeman argues that these linguistic skills share a substrate with insight because they require the brain to make a set of distant and original connections. Although most of us have probably never used age, mile and sand in a sentence before, the aSTG is able to discover the one additional word (stone) that works with all of them. And then, just when we're about to give up, the answer is whispered into consciousness. "An insight is like finding a needle in a haystack," Beeman says. "There are a trillion possible connections in the brain, and we have to find the exact right one. Just think of the odds!"
It took a few days to adjust to the quiet of Woodstock. Dylan was suddenly alone with nothing but an empty notebook. And there was no need to fill this notebook – Dylan had been relieved of his creative burden. But then, just when Dylan was most determined to stop creating music, he was overcome with a strange feeling. "It's a hard thing to describe," Dylan would later remember. "It's just this sense that you got something to say." What he felt was the itch of an imminent insight, the tickle of lyrics that needed to be written down. "I found myself writing this song, this story, this long piece of vomit," Dylan said. "I'd never written anything like that before and it suddenly came to me that this is what I should do." Vomit is the essential word here. Dylan was describing, with characteristic vividness, the uncontrollable rush of a creative insight. "I don't know where my songs come from," Dylan said. "It's like a ghost is writing a song." This was the thrilling discovery that saved Dylan's career: he could write vivid lines filled with possibility without knowing exactly what those possibilities were. He didn't need to know. He just needed to trust the ghost.

This was a staggeringly strange way to create a piece of pop music. At the time, there were two basic ways to write a song. The first was to be like the Bob Dylan that Dylan was trying to escape: compose serious lyrics on a serious topic. The second way was to compose an irresistible jingle full of major chords. Such predictability is precisely what Dylan wanted to avoid; he couldn't stand the clichéd constraints of pop music. And this is why that "vomitific" writing was so important: Dylan suddenly realised that it was possible to celebrate vagueness, to write lines that didn't insist on making sense. He would later say that Like A Rolling Stone was his first "completely free song... the one that opened it up for me".

In retrospect, we can see that the composition allowed Dylan to fully express, for the first time, the diversity of his influences – Arthur Rimbaud, Fellini, Bertolt Brecht and Robert Johnson. There's some Delta blues and "La Bamba", but also plenty of Beat poetry, Ledbetter, and the Beatles. What Dylan did was find the strange thread connecting those disparate voices. During those frantic first minutes of writing, his right hemisphere found a way to make something new out of this incongruous list of influences, drawing them together into a catchy song. He didn't yet know what he was doing – the ghost was still in control – but he felt the excitement of an insight, the subliminal thrill of something new. ("I don't think a song like Rolling Stone could have been done any other way," Dylan insisted. "You can't sit down and write that consciously... What are you gonna do, chart it out?")

Those six minutes of raw music would revolutionise rock'n'roll.
The constant need for insights has shaped the creative process. In fact, these radical breakthroughs are so valuable that we've invented traditions and rituals that increase the probability of an epiphany, making us more likely to hear those remote associations coming from the right hemisphere. Just look at poets, who often rely on literary forms with strict requirements, such as haikus and sonnets. At first glance, this writing method makes little sense, since the creative act then becomes much more difficult. Instead of composing freely, poets frustrate themselves with structural constraints. Unless poets are stumped by the form, unless they are forced to look beyond the obvious associations, they'll never invent an original line. They'll be stuck with clichés and conventions, with predictable adjectives and boring verbs. And this is why poetic forms are so important. When a poet needs to find a rhyming word with exactly three syllables or an adjective that fits the iambic scheme, he ends up uncovering all sorts of unexpected connections; the difficulty of the task accelerates the insight process.
The story of Like A Rolling Stone is a story of creative insight. It took only a few seconds before a mental block became a work of art. In a 1966 interview with Playboy, Dylan assessed the impact of the sudden breakthrough on his music. "Last spring, I guess I was going to quit singing," he said. "But Like A Rolling Stone changed it all: I didn't care any more about writing books or poems or whatever. I mean, here was something that I myself could dig."
What Dylan dug was the strangeness of the song, the way it sounded like nothing else on the radio. In that lonely cabin, he found a way to fully express himself, to transform the fragments of art in his head into a new kind of song. He wasn't just writing a pop single – he was rewriting the possibilities of music.

• This is an edited extract from Imagine: How Creativity Works, by Jonah Lehrer, published on 19 April by Canongate Books at £18.99. To order a copy for £15.19, including mainland UK p&p, visit the Guardian Bookshop. Jonah Lehrer will be discussing creativity as part Canongate Talks series in Edinburgh on 28 April.