top of page
Summary
BRINGING HISTORY TO LIFE
The New York Times Research & Development Team explores how emerging technologies can be applied in the service of journalism. In the winter of 2020, a proposal I co-created for an immersive history experience was selected as one of six finalist projects for the Spatial Computing Challenge, hosted by R&D and the NYC Media Lab.
For the four months that followed, we worked closely with mentors from the Times to develop our vision for a future in which modern and historical reporting can coexist across three dimensions in digital and print.
The Metro ARchive is an immersive augmented reality app that enables users to experience New York City history through the lens of centuries of New York Times journalism.
Users can participate in the experience by visiting Chinatown's Doyers Street, or by engaging with a magazine printout using a mobile device.
Whether you're a tourist, an educator, or a local, the Metro ARchive offers a fresh perspective on streets that deserve a second glance. The experience unites education, entertainment, and social justice by putting the narrative at the forefront.
Duration
1/20 - 4/20
Tools
Illustrator
Photoshop
QGIS
Blender
SketchUp
Skills
User Experience Design
User Interface Design
Graphic Design
3D Modeling
Team
Sammy Levin (Design Lead)
McCoy Zhu (Software Lead)
Dalit Shalom (Product Design Mentor)
Yuri Victor (Software Engineer Mentor)
Multiple contexts, One unified experience
On-site context
Users can engage in a guided tour of Manhattan's Chinatown by aiming their phone at historic sites. 3D modeled buildings are superimposed overtop the camera feed, transforming archived images into interactive environments.
Print context
To amplify reader engagement in the New York Times Magazine, the Metro ARchive platform places digital assets in conversation with physical articles by visualizing the past, present, and future of varied sites.
Bring the streets and buildings of old New York to life before your eyes
Visualize history from every angle
Three viewing modes provide a range of ways to interact with content, both on-site and in print. In a quick swipe of the mode selector, New York Times articles written hundreds of years apart are now put into a whole new conversation.
Immersive mode
Get up close and personal with history by viewing a full-scale overlay of a historic building.
Historic Mode
Overlay a historic narrative with relevant photos and excerpts
from the NYT Archives
Contemporary Mode
View articles and visuals that illustrate the neighborhood's change over time
Get lost in the story
The print version of the Metro ARchive empowers readers of the New York Times Magazine to experience the narrative in a new dimension.
Graphics and layout were created with readers of varying curiosity and technology comfort levels to read and explore at their own pace.
The core of this article is its spatial component. Understand Chinatown's layout by tearing out the magazine spread and circling the square board as if you were playing a game of Monopoly.
By reading information in print, and interacting with the map and buildings in 3D, digital and print mediums each do what they do best.
The Metro ARchive celebrates print journalism by building around it, not over top of it.
Immersive mode
No New York? No problem.
The print version also showcases rich historic architecture in contrast with its modern equivalent, from the comfort of your coffee table. As project development pivoted to remote during the Coronavirus Pandemic, so too did the project itself. By building at street scale and table scale, the full experience is accessible to all.
Historic Mode
View a miniature 3D preview of the historic model for the selected building
Contemporary Mode
See the same site transformed by centuries of development at the swipe of a fingertip
The Metro ARchive takes the principles of augmented reality and applies them in ways that allow users to experience history immersively, enrich everyday environments, and contextualize current events
Iterative growth
A key step in the early ideation cycle was deciding which interactions and assets should exist in the UI, the 3D scene, or in physical reality.
Two key learnings were identified at this stage. Text and graphics are far more visible when printed instead of viewed within the 3D space, and the best 3D interactions are the ones that are spatially intuitive to the user.
To develop an authentic representation of the street layout, I leveraged QGIS and Blender to create an extruded slice of the street which references satellite and survey data for accurate building heights and floor plates.
This model exists in the virtual scene and is constructed of smaller building-level models, which can be selected and highlighted if they are part of the narrative.
An earlier version of the board featured text in the virtual space and richer imagery, labels, and colors. After testing this prototype, we chose to pare back on dense visuals on the board in favor of including them in the digital experience.
The result of these findings was a set of interactions that work without explanation, and graphics that are just as clear as they are inviting to read and inspect.
Our four months of development were laden with trips to and from Doyers St. for site visits, photogrammetry captures, and play testing. This was accompanied with days spent deep digging through photo archives and historical Times articles to piece together the narrative of what happened and how it was portrayed. One theme that was challenging to navigate was media sensationalism at the time. Because of the presence of gang activity in parts of Chinatown, Chinese immigrants were often disproportionately portrayed as enemies of law and order in historic New York.
Intro the third dimension
One of the most challenging portions of this project was translating archived imagery of buildings into a range of 3D, AR-friendly assets. This meant working across SketchUp, Blender, and Photoshop to develop a flexible pipeline for asset creation.
1
2
.png)
3
With a long focal length image of the building, it's possible to closely match perspectives between a reference image and the 3D space in SketchUp, which makes modeling a breeze.
Textural details were extracted from archived photographs, or sometimes inferred based on context. A range of historical photos from different angles informed the texture design.
All the pieces came together in a textured 3D model that was packaged in a way that makes it delightful to explore at tabletop scale, and at street level in the virtual scene.
LESSONS LEARNED
This project granted me incredible access to world class resources and guidance, and was an exciting venture into what digital product design looks like in 2D and 3D. In addition, learning from mentors at The New York Times has sparked a deep passion for shaping the future of digital journalism.
-
I enriched my research skills and methodology
-
I made strategic design decisions based on usability research and product objectives
-
I worked across a range of features and surfaces
-
I learned from the constructive criticism of mentors and peers and set new goals at each checkpoint
-
I speculated the implementation of a product that integrates into a broader portfolio
Further documentation
Press mentions
Working with Universities to Experiment with Spatial Computing
- The New York Times
See Old New York Appear Before Your Eyes with this New AR App
- Time Out New York
New Application Brings New York City to Life
- immersive-technology.com
New Augmented Reality App Recreates Historic NYC Buildings
- Untapped Cities
- AR Planet (Taiwan)
AR App Shows What New York Used to Look Like
- VRRoom
If You're Interested in Urban History, Try the Metro ARchive
- New York University News
bottom of page