Image credits: poetry camera
Have you ever stood in front of a sequoia and thought, “Wouldn't it be great if there was a poem instead of a tree?'' Neither did Joyce Kilmer. But Kelin Carolyn Zhang and Ryan Mather set out to bridge the gap between AI technology and poetry with their fascinating brainchild, Poetry Camera. This open-source device combines cutting-edge technology and artistic vision to create a work that pushes the boundaries of both fields.
At first glance, Poetry Camera looks like just another gadget in an ever-evolving line of digital devices. But a closer look reveals that this is no ordinary camera. Rather than simply capturing images, Poetry Camera takes the concept of photography by generating thought-provoking poems (or as thought-provoking poems as AI poetry can get) based on the visuals it encounters. Take it to new heights.
The project's masterminds, Zhang and Mather, have seamlessly blended their expertise in technology and art to create a technically impressive and aesthetically captivating device.
Poetry Camera's story began as a personal passion project for Zhang and Mather, fascinated by the idea of combining their love of technology with their appreciation of art. In the early stages of development, they spent countless hours tinkering with different components and experimenting with different AI models to bring their vision to life.
“This project started with me accessing GPT-3. I'm a nerd, so my first instinct was to play Dungeons & Dragons. I thought, 'Wouldn't it be great if I could play Dungeons & Dragons with this?' I thought. And yes, it worked. This was back in the day when you had to do quick engineering. So it took some effort to get it working. But I also had the idea of building a camera as a project,” says Mather. “What if I took my camera and it was a reaction to Instagram culture? What if text came out instead of pictures? …Because everyone prefers the book version to the movie, It’s like that to capture a moment.”
As Zhang and Mather refined their prototype, they began sharing their work with friends and family at social gatherings. The response they received was amazing. People were fascinated by the concept of a camera that could generate poetry based on what it saw. The device immediately became the center of attention, sparking lively debate and igniting the imagination of everyone who encountered it.
These initial reactions provided strong motivation for Zhang and Mather to continue improving their invention. They realized Poetry Camera had the potential to bridge the gap between technology and art in a way that resonated with people. Once word of the Poetry Camera spread, the duo was inundated with inquiries from curious people who wanted to experience the device for themselves. This interest led them to consider the possibility of converting their project into a potential commercial product and making the magic of Poetry Camera accessible to a wider audience.
technology
At the heart of this innovative device is the Raspberry Pi, a single-board computer the size of a credit card that packs a powerful punch. This small but powerful component acts as the brain of Poetry Camera, allowing it to capture images and communicate with OpenAI's GPT-4 to generate poems.
The Raspberry Pi captures images and uses computer vision algorithms to analyze the visual data. An AI model then interprets the image and identifies important elements, colors, patterns, and emotions within the frame. This information serves as the basis for the poem generation process.
Zhang turned to the camera during a Zoom interview.
Then the following poem appeared:
The man on the other side of the lens adjusted his gaze,
Within a square in pixel space.
Zoom in on faces stored in virtual locations,
It's been a long time since I left the rat race.Sunlight shines through the faded thin fabric,
He holds a lukewarm cup of coffee in his hand.
Conversations drift across virtual frontiers;
His image is reflected in the mirror and appears on the screen.Books were leaning on the shelves, and people were chatting quietly.
A thriving plant, the only living thing.
Wearing glasses, the house becomes his gentle batter,
His world was scattered within these walls.This morning in April 2024,
A new daily life that you can quietly acquire.poem by @poetry.camera
Mr. Chan theorizes that the phrase “breaking out of the rat race” may be referring to my Hawaiian shirt.
The inventors say Poetry Camera's output is not limited to a single format. Because the device is open source, users can choose from a variety of poetry formats, including haiku, sonnets, and free verse, depending on their preferences and their ability and willingness to engage with the source code.
As Poetry Camera continues to evolve, Zhang and Mather are exploring new ways to enhance its capabilities. They constantly refine and select AI models, enhance device prompts, and experiment with advanced natural language processing techniques. The goal is to create devices that generate poetry, engage in meaningful dialogue with users, and foster deeper connections between technology and art.
The intersection of art and technology
Poetry Camera is a testament to the incredible possibilities at the intersection of art and technology. By harnessing the power of AI and machine learning, Zhang and Mather have created a device that not only captures the beauty of the world around us, but also interprets it in amazing and inspiring ways. Although the technology is simple, I love the accessibility that Poetry Camera provides. It becomes so much easier to dream and push the boundaries of what technology can achieve.
Screens dominate our daily lives. Poetry Camera offers a refreshing break from the norm. Rather than relying on digital displays for poetic output, the device employs a more tactile and immersive method of interaction. The generated poems are printed on paper, physically embodying the artistic experience. The ephemeral nature of art is captured to a great extent by Poetry Camera.
“We don't store images or poems digitally. There are several reasons for that. One: It's easier. Two: Privacy. Third: Poetry is such an ephemeral type of thing. When it's like a work of art, it adds a special meaning to the poem. If you lose that, it's gone,” Mather explains. “Nowadays everyone has a camera in their pocket and uses a cell phone. We wanted to do something completely different.”
Products like Plaud and Humane Ai Pin have seen new UIs, and this device continues in the same vein. This frees users from constant visual stimulation and allows them to engage with their Poetry Camera work in a more mindful and meditative way. Although this may not be true for all art students, the visceral experience of holding a printed poem in your hands encourages you to ponder the words, appreciate the beauty of language, and form a deeper connection with the work of art. .
Design philosophy extends beyond the device and impacts the entire user experience. Chan and Mather have carefully crafted Poetry Camera to foster a sense of wonder and discovery. Capturing an image, waiting for the AI to generate a poem, and then receiving the printed output creates a sense of anticipation and surprise, increasing overall engagement with the device.
Reflecting the future of AI and personal computing
Zhang and Mather's design philosophy for Poetry Camera reflects their broader vision for the future of personal computing and AI. They believe that technology should be seamlessly integrated into our daily lives, enhancing our experiences without overwhelming us with constant stimulation. By creating devices that prioritize simplicity and artistic expression, they are challenging the status quo and paving the way for a new era of technology.
“I think poetry cameras are a microcosm of what's going to happen to many industries with AI. Poetry cameras are different from traditional fields. You can't say, 'Oh, I'm a poetry photographer for corporate events. ”, laughs Chan. The Poetry Camera sits between this established field of photography and poetry. It's something new and strange. Most importantly, it's fun to watch people interact with them. People find childlike joy in their playful personalities. ”
What's next?
As Poetry Camera continues to captivate audiences and generate buzz, Chan and Mather are carefully considering the future of their innovative work. Although the device began as a personal passion project, the overwhelmingly positive response led us to explore the possibility of commercializing it for a wider audience.
“After the first 100 questions, we said, ‘We’re not selling it,’ but after the 101st question, we started thinking about whether we should offer it to people and started looking more closely at it.” Zhang thinks. “But at the same time, this is an art project, right? Our first response was to take capitalism out of it.”
But true to their commitment to sustainability and conscious consumption, Zhang and Mather are taking some time to rush into mass production. Instead, they're taking a more measured approach, prioritizing quality over quantity and allowing for the possibility of limited product drops. By carefully controlling our production process, we aim to ensure that each Poetry camera is manufactured with the utmost attention to detail and meets high standards for performance and durability.
I insisted to my team that we needed to purchase a hardware product. The phone has a camera, internet connection, and screen. Wouldn’t it be more sustainable to use existing hardware?
“The most sustainable option would be to not make anything at all,” Chan agrees. “But we initially started this as an open source DIY project and got all the instructions online. It's just that it's difficult to gather all the different parts that we need. Initially, we… was considering the idea of creating a cardboard casing, but part of what makes people love it and want to play with it is the beauty of the Poké Ball polished product. I realized that for the Poetry Camera to work to its full potential, it needs to be a sleek looking product, and I'm not sure how to approach it with sustainability in mind. Yes, it is important to us.”
Looking to the future, Chan and Mather envision a future where the poetry camera is more than just a novelty device, but a catalyst for changing the way we interact with technology and art. They hope their work will inspire others to explore the intersection of these two fields, pushing the boundaries of possibility and creating new forms of expression.
I love living in a world where enthusiast hobbies intersect with art and technology, and this project fits right into that Venn diagram.
“For me, this continues to be about expression. This is a way to shape the world we want to see, a world where people can play with this new technology. It's not all bleak. And it's not just about AI imitating humans in deeper, faster, and worse ways,” Zhang concludes. “It's about using new technology to create something completely new and reawakening your inner child. This project isn't about making money, it's about making that childish wonder possible again.”
If you want a closer look at how the camera works, TikTok seems like the best resource.