Bronze Idea and Conceptual Design · 2024
ReCollection Interactive AI Art Experience Design
by Weidi Zhang and Jieliang Luo
- A' Design Award 2024 Bronze Winner in Idea and Conceptual Design
- Awarded April 15, 2024
- Estimated Number of Design Views: 82,572
- Verification code 48D4A31111 · Verify this award
About ReCollection Interactive AI Art Experience Design
ReCollection is an interactive art installation that forms images of personal memories from spoken language. Participants whisper a fragmented memory into a microphone. An artificial intelligence system converts the speech into text, completes the sentence as a short narrative, and generates images from the narrative. The images are then processed algorithmically into an evolving real-time visualization. The installation explores the relation between language, memory and imagination, and is also intended as a prototype for future work with people affected by dementia.
Press Kit
- Unique Properties
- ReCollection is an AI art experience that synthesizes personal memories based on users’ language input, blurring the boundaries between remembrance and imagination through AI system design and experimental visualization. Beyond its potential as a future therapeutic prototype for dementia groups, this work provides new possibilities for collective memories and cultural reproduction.
- Dimensions & Packaging
- Space Dimensions: While adaptable to different room sizes, we suggest a space dimension of 7m x 7m x 4m (L x W x H) to achieve the desired projection size and quality.
ReCollection was displayed as an interactive art installation incorporating a wall projection or an LED screen, and a pedestal positioned to face the projection. This pedestal was designed to hold a microphone and a USB button for interactivity. Our exhibition space was intentionally designed in a dark environment, lit by the projection light and a dimmed spotlight situated atop the pedestal.
Technical Properties:
1. Audio Setup: A speaker tailored for the audio component.
2. Main Pedestal: This will house both the USB microphone and a USB button.
3. Instruction Pedestal: A separate pedestal dedicated solely for the placement of instructions.
4. A USB Microphone
5. A PC
6. A 4K capable projector. - Tags
- generative design, data visualization, interactive AI art
- Production Technology
- ReCollection integrates AI systems and experimental visualization into the design of an interactive art experience. By employing AI techniques like speech recognition, text auto-completion, and text-to-image models, it translates voice inputs of fragmented language into moving computer graphics in real-time. In the art installation, ReCollection invites participants to share their personal memories in one sentence. The AI system then transforms speech to text and automatically fills in details as a completed narrative, generating new visual memories.
- Design Challenge
- In ReCollection, AI is engaged in the decision-making process in imaginative acts. Presently, AI's role in dementia research spans from early detection and progression mapping of the disease to serving as companion for individuals with dementia. However, the utilization of AI in dementia research often remains confined to caregiving roles, rarely exploring its potential in more symbiotic, imaginative collaborations. Drawing inspiration from the development of dementia research and its critical turn, ReCollection investigates connections among language, memory, and imagination through integrating AI as a non-human narrative agency in collaborative worldmaking, investigating the artistic expression of memory and narratives.
We have developed an interactive AI-powered story-generation system, comprising three major components that synergize to create a comprehensive narrative experience, each contributing to a seamless storytelling process.
The system begins with Whisper an advanced speech-to-text model developed by OpenAI, capable of transcribing spoken words into written text. Significantly, Whisper supports 98 languages. For the second component, we employ GPT- 4 for text completion. Drawing inspiration from documentaries on Alzheimer's, such as A Marriage to Remember. we've defined two distinct agents in the prompts. The first agent controls the content, ensuring the narrative is coherent and contextually relevant, while the second agent influences the visual style, akin to how a camera lens or shooting angle might affect the visualization in a film. This dual-agent approach effectively creates a storyboard in written form, adding a layer of depth and sophistication to the generative narrative.
Lastly, our system integrates Stable Diffusion APIs provided by Replicate. Stable Diffusion is a text-to-image model, to convert the written narrative into vivid visual representations. This not only enriches the storytelling with illustrative content but also ensures that the visual output is as engaging and immersive as the written story. Collectively, these components synergize to create a dynamic narrative experience for creative expression. We notice that our system can culturally adapt the visual output to match the input language. This allows for a nuanced and culturally sensitive storytelling experience, enhancing the relevance and immersion of the narrative. Leveraging the collective strengths of each AI model, ReCollection produces a personalized and engaging user experience that pushes the boundaries of traditional storytelling. - Project Duration
- Duration varied (Interactive work)
Location: this project has been presented in Los Angeles Convention Center (United States), Signal Immersive Art Gallery (Canada), Media and Immersive eXperience Center (United States), and will be presented at Cinema Mystica (Hungary) - Operation Flow
- In the art installation, a participant will whisper fragmented memories into the microphone. The AI system will automatically fill in the details of the spoken words to complete the text with a narrative using the GPT-4, a large language model. The completed narrative will be sent to Stable Diffusion to generate synthetic images representing the memories based on the machine’s interpretation. The images output by machines is further developed and visualized algorithmically as an evolving interactive experience.
- Research
- This artwork was born of witnessing my grandmother's memory regression due to dementia, where her cherished stories dissolved into fragmented words. Dr. Mary Steedly once described memories as a "densely layered, sometimes conflictual negotiation with the passage of time", and in 2022, over 50 million people faced this painful reality of memory loss due to Alzheimer's and related dementias. Yet, amidst this poignant backdrop, the emergence of text-to-image AI systems in 2022 offered a glimmer of new perspective, as they harnessed the power of language to imagine and reassemble fragmented memories, possibly to weave what time and disease had stolen.
When we coexist with machines, will we accumulate synthetic recollections of collective symbiotic imagination?
Is language capable of re-weaving and synthesizing memories?
How does our collective memory inspire new visual forms and alternative narratives?
Recollection is an assemblage of intimate human-machine artifacts that emphasizes the contributions from three sides: artists, machines, and participants. This customized AI application facilitates multiple AI techniques, like speech recognition, text auto-completion, and text-to-image, to convert language input into image sequences of new memories. As an interactive experience, participants will whisper their personal memories with fragmented sentences, and our system will automatically fill in details, creating new touching visual memories.
We developed our customized AI system by fine-tuning a pre-trained transformer-based AI model to learn the documentaries of Alzheimer patients’ visual memories and their descriptions. The system imagines new memories of "love" and "loss" by interpreting real-time narratives from participants in the installation. Our system emerges as a vibrant and inclusive conversation starter, transcending boundaries with support for over 89 different languages, embracing the diverse cultural artifacts.
We did not employ the generative visual output from our AI system as our final delivery in the installation. Instead, we draw inspiration from Monotype, a printmaking technique with a history that dates back to the 1640s, which produces unique visualizations on paper through printing and reprinting. Therefore, we program our system to automatically transform the machine-generated visuals algorithmically into an evolving real-time visualization with a unique aesthetic through image processing and experimental data visualization.
By providing a conceptual framework for non-linear narratives, which constitute symbiotic imaginations, and future scenarios of memories, culture production, and reproductions. It may inspire the cure for memory regression by providing a future scenario, a thought experiment, and an intimate recollection of symbiosis between beings and apparatus. It raises people's awareness of future memory preservation and their empathy for the dementia community through a personalized aesthetic experience. It offers an artistic approach and future prototype for cultural heritage reproduction and re-imagination and explores the tensions that exist in the co-relations between visual representations, language, and narratives. - Inspiration
- ReCollection was born of witnessing my grandmother's memory regression due to dementia, where her cherished stories dissolved into fragmented words. Since 2020, over 50 million people have faced this painful reality of memory loss due to Alzheimer's and related dementias. Yet, amidst this poignant backdrop, the emergence of text-to-image AI systems in 2022 offered a glimmer of new perspective, as they harnessed the power of language to imagine and reassemble fragmented memories.
- Image Credits
- credit to Weidi Zhang and Jieliang Luo
Colors of ReCollection Interactive AI Art Experience Design
Grand Interview about ReCollection Interactive AI Art Experience Design
Witnessing my grandmother’s memory fade was the beginning of ReCollection. I watched her cherished stories slowly collapse into fragments of language. She could still feel the memories, but she could no longer describe them.In 2022, when text-to-image models emerged, I realized that even incomplete language could become a creative pathway. The technology revealed that fragments do not need to be corrected or restored. They can be transformed into full visual narratives.ReCollection builds on this idea. It turns broken speech into imagery that helps people experiencing memory loss visualize what they can no longer articulate. It also offers the general public a space to speak about fading memories and to see how a machine interprets them. The imaginative details generated by the system become an entry point that connects imagination with remembrance, offering a gentle way to engage with memories that are slipping away.
Supporting many languages is essential because people respond emotionally in the language they grew up with. During exhibitions, we often saw participants become teary-eyed when the system generated visuals based on their own language. It made the memories feel personal and culturally grounded.The AI system itself is also shaped by culture. When someone speaks in Japanese and mentions Tokyo, the generated visuals naturally carry Tokyo-specific elements. The characters might appear more Asian, and the environment reflects the cultural context embedded in the language. Different languages subtly guide the tone, look, and atmosphere of the synthesized memories.This inclusivity is crucial. ReCollection is meant to be a shared world-building experience, not something limited to a single group. By allowing people to speak in their own languages and see their cultural cues reflected in the visuals, the project becomes a universal space where participants from any background can contribute their memories.
Drawing inspiration from Alzheimer’s documentaries such as A Marriage to Remember, we designed the system around two distinct agents. One agent focuses on content, making sure the narrative remains coherent and emotionally grounded. The second agent shapes the visual style, much like how a camera lens or shooting angle influences the mood of a film.Together, they create a written storyboard that guides the generative process. This dual-agent structure adds depth and cinematic nuance to the storytelling, allowing the final output to feel both narratively meaningful and visually expressive.
ReCollection is not designed to solve a specific challenge faced by individuals experiencing memory loss. The project comes from a speculative design mindset rather than a problem-solving framework. The goal is to create an alternative reality that might inspire new ways of thinking about memory and its future, rather than offering a direct therapeutic tool.It sits at the intersection of contemporary design and conceptual art, and it is intended as a universal experience for the general public. While we would love to present the work to people living with memory loss and learn how they respond to it, that is not the central objective of the piece.At the same time, the project resonates with ideas from medical humanities, especially Jenkins’s concept of the “reimagined memory” as an inter-embodied self-montage. Through interaction, participants see how their memories continue to shift, accumulate, and reshape their sense of self. ReCollection reflects this evolving selfhood rather than attempting to stabilize or correct it.
One of the most unexpected discoveries came from the imaginative details generated by the large-language model. We use GPT-4 to enrich participants’ fading memories, since many inputs are just brief descriptions with missing context. But when the AI fills in those gaps, its cultural and social biases surface very clearly.For example, when someone mentions doing housework, the generated characters often default to women. These moments reveal how the model mirrors the biases embedded in human language and society. Seeing those assumptions appear inside a “memory reconstruction” process becomes a reminder that we must remain cautious when working with AI.At the same time, new media art can serve as a platform to expose these issues. By allowing these biased or unexpected details to surface, the piece invites audiences to reflect critically on the systems that shape both AI outputs and our own cultural narratives.
In the installation, we intentionally chose not to display the raw AI-generated images. Instead, we drew from traditional fine-art practices to create a more textural and expressive visual language. Monotype printmaking, which dates back to the 1640s, inspired the idea of images that exist only once, with each impression carrying traces of pressure, ink, and chance. We also looked to slitscan photography, which captures time as stretched, distorted slices.By merging these techniques with generative methods, ReCollection explores an aesthetic of memories that dissolve, tilt, smear, and reprint themselves over time. We were less interested in producing a clean digital output and more interested in visual fragments that echo the instability and poetic impermanence of human memory.
I do see therapeutic potential in ReCollection. Because the system transforms spoken fragments into visual stories, people experiencing memory loss may be able to see these images and reconnect emotionally with moments from their past. The experience is intuitive and multisensory, which can make the memories feel more accessible and emotionally resonant.This relates loosely to the principles of reminiscence therapy, which uses prompts such as photographs, music, or familiar objects to evoke personal memories. While ReCollection is not designed as a clinical tool, it shares a similar intention: to create a space where memories can surface through gentle sensory cues. In our case, the cues are generated by the machine, shaped by the person’s own voice, and reimagined visually. The goal is not treatment, but to offer an alternative, imaginative way for people to engage with memories that may otherwise feel out of reach.
In the experience, participants speak a single sentence about a 3D memory within seven seconds. GPT-4 then expands that fragment into a fuller narrative by adding imaginative and emotional details. That story is passed to Stable Diffusion, which we fine-tune using our dual-agent approach so the resulting visuals carry a sense of love, loss, and nostalgia.This entire pipeline makes the system feel more empathetic. The visual model was also fine-tuned with references from Alzheimer’s documentaries and patient-described visual memories, which helped shape how the system interprets themes of fading memory. Combining GPT-4’s narrative sensitivity with Stable Diffusion’s emotional visual language allows ReCollection to reconstruct memories in a way that feels culturally aware and emotionally grounded.
The ReCollection series is essentially complete. The main future development is the release of the AI film You Only Have 7 Seconds, which recently won Best AI Art Video at the Burano Artificial Intelligence Film Festival in Venice, Italy. This film represents the newest chapter of the project. It is created from the archive of voice inputs collected during the international exhibition tour.We curated and composed this material into a cinematic montage that highlights the multiplicity of human expressions. The film becomes a kind of collective chorus, revealing both the shared emotional threads and the striking differences across participants’ memories. It’s less about achieving technological “resolution” and more about creating a collaborative world-building experience.In that sense, the ReCollection series has reached its conclusion, and the film stands as the final synthesis of everything the project set out to explore.
Exhibiting ReCollection internationally became even more interesting once we began creating the film You Only Have 7 Seconds from the archive of collected inputs. When we reviewed the dataset, we noticed clear patterns in what people choose to recall: family bonds, personal milestones, food, and small everyday moments. These themes appeared repeatedly across countries and cultures.At the same time, the details were always different. Each person’s cultural background, identity, and lived experience shaped the emotional texture of their memory. So we saw both the deep commonality in what people value and the beautiful differences in how they express it.This duality is what inspired the film. It celebrates the multiplicity of human expression by turning intimate whispers into a shared world-building experience. Even though the memories come from many voices, together they form a collective portrait of how diverse and yet deeply connected we are.