Lately, I’ve been thinking of growing up in Poland, and it struck me that I was exposed to some rather heavy literature and poetry at quite a young age. As a kid, I remember liking a 19th-century poem called “Marionettes” by Cyprian Norwid that centered on boredom (perhaps because I liked the irony of being a bored 4th grader trying to read and understand complex words).

While my teacher didn’t quite share my literary choices, she did enjoy my hand-illustrated assignment where each verse had a small picture of what “showed up in my brain.” I thought a fun experiment could be to re-do this (long-lost) assignment in the form of a video, where I work with AI-generated content from the same poem for as many aspects of it as I can. 

So to my 4th grade teacher: Ms. Koslewska, I hope you are reading, this two-minute video is for you:

A Million Choices and Boredom

I started with Runway — a text-to-video generator. Inputting lines from the actual poem led to some interesting imagery such as “a million 
Silent stars shine around the world.” 

The glow looked almost like vintage Soviet-style animations I remembered from my youth: could it be that the machine was attuned to my wants? More likely, it was simply a coincidence. The poem made me realize how closely I identified with the idea of boredom due to too many choices… When everything is possible, is anything interesting? With so much choice (and ease of creation) will things like a moving image of stars elicit little to no emotional response? Perhaps in the future—for now, I was happy to put the video on a loop and meditate thinking way back to my fascination with the stars and the moon as a kid. 

A Victorian-Style Malaise

The prompts for the next lines like “Still the earth, the eons vast” brought up some fun imagery of astronauts (or cosmonauts if you lived in the Soviet bloc). To keep things consistent, I decided to pursue my theme of 19th-century Poland by prompting the program to imagine Polish Victorian cosmonauts. They ended up looking peaceful and asleep (maybe because of the stillness in the poetry line?), and the abstraction on some of the frames intrigued me:

The more video I generated, the more satisfied I was with the context. The echoes of the poem’s intentions were the perfect framing device for the world’s current obsession with AI. The exported footage was extremely impressive, but as the novelty wore off with time, I found myself attempting to art-direct the exports into my vision. To counteract falling into a familiar creative territory, I decided to keep some of the more unsettling, almost human forms to ground the viewers in the fact they are looking at computer-generated imagery. To me, it is an aesthetic choice in its own right.

“I Cannot Do That Hal” — A Machine-Generated Voice

Once the visuals came together (more or less one per line of the poem), I needed a structure for editing. At first, I wanted to have someone simply record a voice-over of the poem, but I was curious about text-to-voice generators, such as Eleven Labs Speech Synthesis. 

The voiceover was not as resonant as one done by an actor, however, the slight detachment in the Nicole artificial voice preset was an interesting way to bring the detachment in the poem to the forefront. I did take liberties in Adobe Premiere where I edited the voice and visuals by speeding up or slowing down Nicole’s delivery, as well as adding a slight reverb to up the “Thoughtful and Spiritual” vibe. 

Norwid Unplugged: Poetry in AI Harmony

For music, I turned to Generative.fm, where I remembered listening to something akin to Hans Zimmer’s Blade Runner soundtrack. To my delight, the creator had an attribution license on their creation allowing me to use it with credit. I was able to generate the exact right timing a few times to get just the right tone and then remixed it in Premiere. 

“I find it so amazing when people tell me that electronic music has no soul. You can’t blame the computer. If there’s no soul in the music, it’s because nobody put it there.” – Bjork

Admittedly, the soundtrack and voice are generated based on a limited data-set, rather than the larger data pools of the video, but it’s easy to see how music and voice could also be text-prompted. As far as I know, there have been some initial explorations in this area, but it is not as refined as the visual tools we have available.

The interesting thing about Bjork’s quote is the notion of putting a soul into an object, like a computer. Does the soul come from the prompts? Is it Norwid? Or is it the folks who programmed it — or perhaps an amalgamation of all the souls working on the output? Or maybe I’m only imagining something there based on my memories.

Echoes and Algorithms

The nicest part of creating the video was the ability to immerse myself in the poem and think of each line. It also made me think about the removal of time limitations on media like music and videos. Perhaps in the future, enjoying a song or a movie will be more like taking a swim - you dip in and stay for as long as you like: and swim fast or just float into sleep. 

Will we simply immerse ourselves in only pleasant experiences and not be exposed to new ideas once these tools become more powerful? Perhaps we already do that when we re-watch a comfortable movie or series, the crafted characters repeating their lines from a series of 0s and 1s bringing us a nostalgic comfort. 

While it’s easy to be cynical about AI and productivity, it’s important to note that this experimental video came together quickly in a day, showing the potential for quick art direction in service of crafting a more polished product. The experiment made me think of the type of deliverables we might craft for our clients: how might we think outside the traditional time-bound narrative of beginning/middle/end, and craft more immersive ways of experiencing data and narratives?

Fade to the Familiar

Familiar shapes and sounds from the screen, a comfortable sofa, and a nice beverage can make for a wonderfully relaxing evening. With new tools that rely on databases of existing media, it might become easier than ever to fall into repeating what is comfortable to us. While writing this article, I started to crave human input solely to hear a different angle on my words, and to be challenged on what I created with AI. After so much text-based input, it was a relief to jump on a call with a colleague who pointed to one of my sentences and said “I’m not sure about what you’re trying to say here.”

For now, I’ll be going back to the ancient technology of old printed books for some relaxation and reflection. 

Credits:
Poem written by Cyprian Kamil Norwid (around 1865–1866), and translated by me. Footage generated with Runway AI and edited in Adobe Premiere.
Voice generated with Eleven Labs Speech Synthesis.
Music created with Generative.fm Music System

Thank you to Brian Pelsoh for being my editor, and helping me clarify my words.
Thank you to all the folks at IDEO exploring the edges of our crafts.