“Big Bill Broonzy had a song called “Key to the Highway." I sang that a lot. If you sing that a lot, you just might write Highway 61.”
AI exists in the space between mimic and maker. And the key to AI's power to learn and create is data. Without data, AI as we know it would not exist. But with a good dataset, it's possible for AI to identify, learn and create on a near-human level. As humans do, AI learns from observation and influence. Many of the AI-powered recommendation algorithms we're familiar with learn by observing human behavior and then generating conclusions on which movies we'll watch or products we're likely to buy. This is not too different from how human artists often gather inspiration for their work. Artists of all genres can name influences, and some influences are more apparent than others in the work created by their admirers.
This is one reason we’re opening up the songwriting catalog of Bob Dylan as we ask AI to generate song lyrics. Dylan is a prolific writer, publishing hundreds of songs and recording dozens of albums in his long career. This creative prolificacy gives us an excellent dataset to work with, with thousands of words for our AI model to study. Dylan's work is also uniquely varied. His lyrical poetry varies quite a bit, from his early days miming Woody Guthrie tunes, to his more recent work which can be densely evocative and opaque.
In addition to the data-friendly characteristics of his output, Dylan’s openness about his approach has always been unique among artists. As referenced in the above quote, Dylan often speaks on the power of influence from other artists and musicians. Speaking about one his most well known original songs, Dylan says:
“I wrote ‘Blowin’ in the Wind’ in 10 minutes, just put words to an old spiritual, probably something I learned from Carter Family records. That’s the folk music tradition. You use what’s been handed down. ‘The Times They Are A-Changin’ is probably from an old Scottish folk song.”
When asked how aspiring songwriters should think about their craft, Dylan says:
"If you like someone’s work, the important thing is to be exposed to everything that person has been exposed to. Anyone who wants to be a songwriter should listen to as much folk music as they can, study the form and structure of stuff that has been around for 100 years."
For a creative exploration of AI, this advice could not be better, and this is exactly what we've done. We've asked AI to spend time learning Bob Dylan's songwriting catalog, to study the form and structure of his writing to see if AI can become a songwriter. Not to mimic Dylan exactly, but to use what has "been handed down" to generate something new. And why not? As Dylan has also said, "Maybe I’ve written enough songs. Maybe it’s someone else’s turn.”
Here's the first song we'll share in this project. I'll share a bit more about the process below the lyrics.
"Hangin' On" by BOB DYLAIN:
They say if you wanna be remembered
You better move fast
You better move with the times
Truman had his White House
And the Capitol was his
The same place that he now sits
I’m hangin’ on to a solid rock
Made before the willow tree
And I’m going to hell for standin’ on a hill
And a—bout it givin’ custody of me
So I got a woman named Marya
She’s a Methodist and she worships at midnight
She pulls out a knife and cuts me in the back
I’ve survived by drinkin’ water
I didn’t make it through the wall crawlin’ by
I’m hangin’ on to a solid rock
Made before the willow tree
And I’m going to hell for standin’ on a hill
And a—bout it givin’ custody of me
Ya know I once wondered what it was all meant to be?
Like a swarm of bees looking for a home
I’m tryin’ to keep from falling down
But like a lot of people that died long ago
It doesn’t make me angry or suspicious
I just can’t remember you
I am not a doctor
I do not experience pain
I do not theorize
I do not speculate
I do not offer my thoughts
To anyone but myself
Pain is not a thought
It’s a feeling
That I’d rather confess
It’s a knowing
That I do not deny
You can feel it in the wound you’ve been healin’
You can feel it in the blood they’ve been drainin’
You can’t turn it off
You can’t remove it
You can feel it in the wound you’re healin’
You cant’ back away
The guilty undertaker pacing on the steps
He’s prayin’ that the sun don’t shine down upon him
The light is in the prisoner’s cell
He’s in the corner prayin’ that the others are asleep
I’m hangin’ on to a solid rock
Made before the willow tree
And I’m going to hell for standin’ on a hill
And a—bout it givin’ custody of me
For those familiar with Dylan's songwriting catalog, it's easy to spot aspects of or even direct quotes from his material throughout his career. "They say if you wanna be remembered / You better move fast," sounds like something from recent albums like Rough and Rowdy Ways as Dylan often reflects on his own personal or American history. "Hangin on to a solid rock" evokes his born-again poetry of albums like Saved. "Guilty undertaker" is a direct lift from his 1966 song "I Want You." While we don't always hear songwriters referencing their earlier work in new compositions, this doesn't seem out of place for Dylan--to call back to a past idea or to suggest that something true 50 years ago is still relevant.
As a whole, this song "feels" like stereotypical Dylan: references to Americana, relationships with women, Biblical or mythological overtones. The nervous anxiety conveyed in lines like "Like a swarm of bees looking for a home," or "He’s in the corner prayin’ that the others are asleep" absolutely feel like they could have come from his original writing. If you were trying to copy Dylan’s style, this is superficially a pretty good attempt.
Is the song as "good" as a Dylan original? I don't think so. One obvious deficiency is in the lack of consistent rhyme. Dylan seems to enjoy song structures that contain pretty strict rhyme while sometimes busting out of that structure just for fun. This song is kind of the opposite--every once in a while you'll find a nice rhyme but generally we're left wanting more, and this doesn't feel like a complete Dylan original without good rhyme. The technical story of how our AI model generates new words might shed some light on this. We're using a model called GPT-2, which was built by observing millions of pages of writing found on the web. Essentially GPT-2 learned to write by hanging out on the internet for way too long (this is clearly dubious territory, discussed at length in this fantastic read). We taught GPT-2 how to write like Dylan by sharing Dylan's catalog with it, but GPT-2 already had some ideas about how to put words together by absorbing a lifetime's worth of varied sources, many or most of which do not contain rhyming verse. So I didn’t expect our model to generate perfect rhymes, and it actually exceeded my expectations in this regard.
But "goodness" isn't exactly the point here, either. What's interesting is how AI can learn enough about a writer to generate new combinations of words that evoke the same feelings and ideas as a human writer. While some of this comes down to simple mimicry, there's also something "creative" about it, similar to how artists are able to connect and express ideas in their own way that compel us to listen, look, or watch. And what's fascinating is just how much of what we might call "creativity" comes from the acts of observation and assembling parts that have already existed--or making use of "data" that's "been handed down." Thanks for reading, and there's more to come.
I’m hangin’ on to a solid rock
Made before the willow tree
Ok, here's a note on my process.
I'm using this dataset of Dylan's lyrics
I'm using this GPT-2 Google Colaboratory notebook
I'm not a coder, so it's been massively helpful to find these free and accessible resources online
When output is generated, I'm doing as little editing as possible. I don't write anything new or alter combinations of words
To make this output as "song-like" as possible, I'm deleting lines that feel like gibberish; I'm editing out lines that are unnecessarily repetitive or superfluous; and I'm sometimes repeating a section of lines to create something like a "chorus" often found is Dylan's music. This is more editing than I would like to do, frankly, but it's the only way to get this output into something like a "song." There are probably other ways, but they're likely out of my technological grasp.
Comments