Google Innovation Grant: The Mind Palace

Quartz was awarded a Google Innovation Grant to create an interactive VR or AR game. We specifically wanted to make an interactive experience that enhanced a story, so Michael Tabb and I worked in tandem to create a game.

The process of creating this on a 4 week timeline while working on other projects was intense, but I really enjoyed it. It required a level of elastic thinking and problem solving that you only really encounter with new technology.

There was A LOT to figure out.

We had to figure out: would we build our own app, or could we try to fit it into the existing Quartz app? What program would we use to build an app? Which one of our long form journalism projects would be appropriate for this platform? What great AR experiences have been created in journalism? Who is our intended audience? How can we make sure that users can enjoy the project via Android, iPhone, and desktop? What do we want this to look like? How can we make sure that the project is small enough to be opened on a phone using data?

The “Memory Palace” was popularized by the show Sherlock as the way that he recalls information. This is a real technique that plays on human psychology that allows people to remember long strings of numbers. It converts arbitrary lists of digits into stories and scenes, which are easier to remember because they're visual and emotional.

AR technology is not standardized across platforms, and doesn’t work in certain browsers, so we ended up creating two separate, but related experiences: one for iPhones and one for Android/desktop users. Above is the Android/browser experience. If you can’t see it, it’s probably blocked by your pop blockers and/or privacy browser extensions, so please switch those off momentarily while you scroll, rotate and click through the project; it’s made to be interactive.

I used the grant funding to take a crash course in Cinema 4D, and applied that training to creating a stylized 3D experience. I art directed, modeled and textured the objects to fit within the Exceptional Humans (paywalled video series) framework.

The concept was to recreate the “Memory Palace” popularized by Sherlock. You take a string of numbers and assign names to them, and create a story. In the Android version, I put the string of numbers on one side, and once you rotate the model, you see the story played out in 3D. Nonsense stories are encouraged, it tends to make them more memorable.

This particular story goes as follows: a BUN jumps over a STICK, knocking another BUN into a GATE. Overhead, a HERO jumps over TWO TREES and lands on a VINE.

The iPhone version (only playable within Safari) is playable here.

Ultimately, I think we created an excellent prototype that would be a great memory improvement tool if it could move through multiple randomized strings of numbers, rather than one example. If you’d like to see the full project, it’s here (paywalled).

Quartz Template

File Size:

My main goal in redesigning the Quartz Video template was to make it easier for video editors to use custom graphics. My first task was to shrink the file size. It was almost impossible to transfer the template to remote workers due to the gargantuan 60 gig size and the many, many duplicate files. Plus, updates were built on top of the old template making file paths for graphic elements very confusing.

Notice the whopping file size of the old template

---

So I remade the whole thing from scratch.

---

LEARNING TO USE MOGRTS:

The text animation is built in, but editors are empowered to change the text size. color, content and position in the essential graphics panel to your left.

I used Adobe's new file type that is specifically designed for motion graphics in Premiere. I created them in After Effects (AE) and use expressions (code within AE that's slightly modified JavaScript) to automate things like color and animation and then create a special file to import that animation in Premiere.

All of our text on screen and graphic elements have to be easy enough to use for someone who is completely unfamiliar with After Effects. Quartz values highly produced content and (for the most part) every team member completes his or her own videos from start to finish. There is no post-production editor, so our template has to be simple, but also flexible enough to work with a variety of editing styles. In order to create such a robust template I did a crash course in using AE expressions and began to learn how to use JavaScript.

You can check the box to create a 3rd line and the blue bars automatically resize to fit the text editors type in.

---

I was able to automate all of our text-on-bar graphics to grow and shrink automatically depending on what the editor types (example below) and improve our credits so no matter how many categories you type in, the spacing is automated. I also significantly reduced the template size to 50 MB by cutting out/condensing several .movs and .pngs, making it easier to transfer to remote workers and use on laptops.