ImagiNense - Building memory palaces using Augmented Reality
An application that supports memorization by helping you create memory palaces.
Role
UX Researcher, UX/UI designer, Prototyper
Project
Individual Project, Cal State East Bay
Tools
Adobe XD, Adobe Aero, Photoshop, Spark AR
Timeline
August 2022 - Dec 2022
Source: Art of memory , wikiHow
The inception of this project came from the famous mnemonic device for long term memory that i used in my studies while growing up. It’s called the method of loci or a memory palace. A Memory Palace is an imaginary location in your mind where you can store mnemonic images. The most common type of memory palace involves making a journey through a place you know well, like a building or town. Along that journey there are specific locations that you always visit in the same order. The location are called loci, which is Latin for locations. (singular: locus, plural: loci)
Introduction
To remember newly learned information, we need new words to integrate into our long-term memory. Various mnemonic devices can help us remember things better.
The Method of Loci is one that has stood out since Greek times. It's a technique that involves using visualizations to help us imagine ourselves walking through a familiar environment while placing vocabulary-related cues along the way. This creates a memory palace in the form of a storyline very unique and personal to us.
The Scope & Problem Statement
Imprinting acquired information uses cognitive function of younger adults for categorizing and organizing knowledge about objects and events into groups based on features.
After a few studies on category learning strategies, the Younger adults (age 20-25) with higher memory capacity use exemplar based strategies for categorizing information.
Even the memory palace method can be a difficult and demanding cognitive task. It involves imagining a space vividly in your mind, which can be challenging for novices. It involves a lot of mental effort to imagine a location and associate symbols to it.
I am highly inspired by MIT Media Lab’s ‘NeverMind’ that uses memory augmentation along with educational interfaces to support human memory, however it is still a study model.
How might we help in consolidation of memory and helping retain information in long term memory by using mnemonic device called memory palace?
I have been passionate about this project mostly because using mnemonic devices for memorizing things have been part of my school days growing up. I still remember one i used to memorize chemistry periodic table, so mnemonic devices helps us in retaining something in our long term memory.
[Reference : Foer, J. Moonwalking with Einstein: The Art and Science of Remembering Everything. Penguin Books (2012), Maguire, E.A. et al. Routes to Remembering: The Brains behind Superior Memory. Nature Neuroscience 6,1 (2002), 90--95]
Project Schedule
Constraints
Technological constraints such as using Vuforia to develop the application in the given timeline was a major constraint.
Creating digital memory palaces using a mobile application.
Finding a generalized list that can be used for the purpose of the prototype.
Proposed Technology
ImagiNense is an interactive application that uses the model of spatial navigation and memory. It applies the Method of Loci that uses the creation of memory palace to learn and memorize a set of vocabularies.
The application i have created has the visual cues that can be markers in surroundings that overlap a 3d view of images.This can be developed using Vuforia Engine 10. It is an augmented reality software development kit (SDK) that enables the creation of AR applications. It uses computer vision technology to recognize and track planar images and 3D objects in real time. Vuforia API can be utilized in Java through an extension on the Unity game engine.
The MIT media lab’s NeverMind supports the outcome of the successful retrieval of complex information using a mental model. Its preliminary user research shows that content memorized with NeverMind (using AR glasses) remains longer in memory compared to general memorization techniques. I aim to base my solution on extending the user research and testing done by them.
Research
Literary Research:
My initial research included reading published research papers on memorization techniques and role of mnemonic devices in consolidating memory. MIT media lab’s Nevermind had a lot of background research around functioning of brain when a user visualizes a memory of moving through a place. Another book that mentions the relationship between memorization and spacial navigation is J. Foer’s Moonwalking with Einstein.
Some of the key points that became the basis of this research were:
1. The mnemonic device memory palace triggers episodic memory for tasks that we normally associate with semantic memory, by using interfaces to passively stimulate the hippocampus, the entorhinal cortex, and the neocortex.
2. Memory palace does the memory encoding by engaging in hippocampal activation and promoting task-specific neural firing.
3. We use semantic memory and encode through repetition, however, by using interfaces there is an opportunity to manipulate the way our brain encodes information and form lasting memories with a single exposure.
Qualitative Research - User Interviews
To validate the memory palace i had an open ended and unstructured interviews where i created a paper pen pretotype of the memory palace and had two exercises. Here my audience was asked to first memorize a list of items by listening to the list vocally said to them only once.
In the second exercise i asked them to memorize another set of items but this time by listening to a story of it and visualizing it. There were other questions around how easy it is for them to visualize while reading a book or watching a movie.
User interviews:
Synthesis of The Interview:
Focus Points:
Normal memory of people : 7-9 out of 15 words for short term memory.
Episodic memory is created through stories that makes the people feel something.
The weirder and the more personalized story for a memory palace, the better the retention in memory
Memorization requires repetition, no matter the technique.
Even visualizing a memory palace in our heads is a time taking task and requires a lot of attention
The list of items to create a memory palace could be a value of a number used in mathematics or a random shopping list that user needs to memorize.
Competitive Analysis:
While there aren’t many direct competitors who have a digital way of creating memory palaces, NeverMind remains the closest and better competitor. It still is not as accessible to users because of the usage of AR glasses and the ergonomics reasons acts as a disadvantage.
The following is an analysis of competitors closest to what i aim to achieve:
Overall Advantages:
Better spacial navigation, 3d visual representation of visual cues for memory palace.
Very unique and to the point background research around Nevermind.
Live memory palace and uses any surrounding the user wished to create a memory palace
Cutting edge technology uses- Augmented reality and affordances of AR glasses.
Helps in memorization and good community at Art of memory Website
Overall Disadvantages:
Most of them are not very intuitive to use.
AR glasses not very accessible and disadvantage of differentiating it from the background
Can be ergonomically very uncomfortable as far as the latest version
Still requires additional application installed on the mobile
Technology specialities required to develop for my purpose of the project.
Not the best overall user journey.
Ideation
My ideation stage majorly came from the paper pretotype stage where i validated the memory palace with my user base and also understood the lists i can include in my project for creating a memory palace for.
"It is a fun technique above all and i will definitely use it in memorizing some big numbers list needed for my class"
Roheet 24, M.S. Student
Initial prototype user testing:
I used the feedback from the user and created a lofi prototype of a memory palace in practice mode for memorizing the value of pi upto 18 digits.
The above prototype validated that digital memory palace . And i further went ahead and added more flows to it and created a hi-fi prototype.
The following the empathy map that helped in adding key features to the application:
Key features:
Onboarding screens helping user understand what a memory palace is and how does it work
Practice items with pre recorded memory palaces that shows how to walkthrough a digital memory palace and memorize
Giving information and walkthrough instructions in every step
Ability to add a new list of items to memorize.
Allow users to create a memory palace by selecting a surrounding in the application itself.
Checking the accuracy of the memorized items in the application itself.
Exploration with augmented Reality:
Tools Used: Adobe Aero and Photoshop
Assets courtesy: Adobe creative cloud.
Adobe Aero is a software platform developed by Adobe that allows users to create interactive Augmented Reality (AR) experiences using 3D models, animations, and various design tools. AR is a technology that enables digital content to be superimposed onto the real world, creating a mixed reality experience that can be viewed through a device such as a smartphone or tablet. Aero allows designers to create AR experiences without needing to write code, making it more accessible for designers and creatives to experiment with AR. With Aero, designers can create interactive AR experiences for marketing, education, gaming, and more, bringing new levels of engagement and interactivity to digital content.
Using augmented reality here helped to create a more immersive and engaging experiences for my users.
By using the camera and sensors on the user's smartphone, the application is able to superimpose digital visual cues onto the real world, making it appear as if the objects are actually present in the player's surroundings.
This adds a layer of realism and interactivity to the application. It also allows he users choose their own surroundings to build a memory palace.
Overall, AR enhances the experience in Imaginense and creates a more exciting and immersive world for Users.
Here are some exploration and ways of using it in the applications:
Final Prototype:
After deciding on the user flows and ideating the screens, i went on to figure out the elements for visual design:
Link to the clickable prototype here.
Onboarding screens:
Practice mode: Learn how to walkthrough a memory palace with rules and steps provided and try memorizing the items.
Use case: Memorizing the value of pi up to 18 digits after decimal.
Test mode: Check the accuracy of the memorized items:
Creator mode: Add your items and create your own memory palace using the library of objects present.
Use case: Memorizing a random list of 6 items
User Testing and Iterations
Goals of the user interview:
Understanding the ease of creating a memory palace.
Adding different kinds of lists: ordered and unordered list.
Understanding the steps to follow and information around memory palaces.
Overall UX.
Summary of insights:
Users took maximum 4 repetitions to memorize the memory palace of up to 18 items .
Most of the participants found it easy to navigate through the application
The information and onboarding steps helped them in the walkthrough of the memory palace.
One of the participant pointed out the unordered list and placing them in the surroundings in unordered manner.
Selection of visual cues could be easier.
Quotes from user testing:
"I like the overall visuals and to add an item in the surrounding"
Sam, 26
"For unordered lists like a grocery list, i would like to place items in any order in the surrounding"
Atish, 26
"Selecting visual cues from library can be easier if the whole box is selected"
Alka, 27
"Watching myself memorize items like such long numbers feels like achievement"
Anna, 28
"I still remember the memory palace from your lo-fi prototype"
Rob, 28
Reflections
User testing is a key aspect at every stage.
I tried including user tests from the pretotype stage itself and it really helped in every next step and defining my specific audience. Even after the lo- fi and hi-fi prototype stage i was able gain insights from user testing.
Audience and time constraints shaped the project majorly.
When i started this project i had different expectations and i only focused on creating a memory palace using augmented reality, but my audience helped me understand some shortcomings of it and thus decided to create a usable prototype in form of a mobile application. This also allowed users to create a memory palace at one place and also a whole application came out with various flows around it.
Adjusting Design Methodologies
My design methodologies also shifted a lot over the 4 months, i used various prototyping softwares like adobe aero, adobe xd and even after effects. But ultimately i used adobe xd for all the prototyping. But it did gave me an understanding of future scopes of the project.
Future Scope
If the UX of Augmented reality progresses enough in the future, the whole application can be converted into a way of creating AR filters which would act as a memory place. Each memory palace would be an AR filter.
Adding Gamification as an incentive for the users. We can add badges every time someone memorizes items and also a community can be created where people can share their score and badges. These would represent the number of memory palaces they memorized. One can also create a general memory palace and share it in the community.