- NSF funding sees an idea bloom into groundbreaking design technology.
- By elevating the act of sketching, Mental Canvas restores ideation to its fruitful place in the creative process.
- New technology provides a seamless transition from 2D to 3D, and a new media form is born.
For Julie Dorsey, sketching is a way to solve uncertainties; it’s a tool for trying out possibilities. Dorsey is the creator of Mental Canvas, a sketching technology that occupies the intersection between 2D drawing and 3D modeling. Thanks to forward-looking investment from the US National Science Foundation (NSF), this new media form may transform the way we design.
Historically, scientists have used sketchbooks to tease ideas apart — they draw because they seek answers. While the digital revolution has enhanced fine arts like music and photography, it has added little value to drawing, Dorsey argues.
Dorsey, a professor of computer science at Yale University, realized that the languishing state of free-hand drawing was actually hurting productivity across several domains. Combining the computer science and design disciplines, she set out to find a more direct way to work with sketched shapes during the crucial ideation stage.
At its core, Mental Canvas is a graphic design system that fuses photographs and hand-drawn shapes to duplicate real world geometry and space. It quickly adapts these 2D drawings into 3D representations, and lets a viewer reposition a sketch to see what’s behind it or to the left or right.
“It’s really applicable to anyone who draws or sketches as a regular part of their workflow,” Dorsey says. “This technology gives us the freedom to try something out and add something to it without having to impose a level of detail or resolution not yet achieved. Coherence is obtained gradually.” Rather than rush to a constraining Computer-Aided Design (CAD) model, Mental Canvas allows designs to evolve in a more continuous way.
Too often, an artist's individuality is lost in digital illustration, but not so with Mental Canvas. Everything remains hand drawn and reflects the hand of the illustrator. A good example is The Other Side, a 3D expansion of Istvan Banyai’s 2005 book of the same name. Available as a downloadable app, The Other Side lets readers pan, zoom, and fly through each illustration – tilting the tablet even moves the picture in true 3D fashion.
The Other Side is the first commercial application of Mental Canvas, and is enough to indicate Dorsey has discovered a new media form. Along with it, she has issued a call to slow down, explore ideas, and not rush into constraints that short-circuit the creative process.
In a time of lightning computation and instant connection, Dorsey’s technology encourages reflection before a rush to production. This parallels the patient fostering afforded by the NSF as her idea came to fruition. Mental Canvas began as a loose idea she wasn’t even sure would go. However, a receptive NSF program director saw potential.
Dorsey initially received a Small Grants for Exploratory Research (SGER) award, allocated for high risk, yet promising research. After further development, her technology had progressed enough for an Early-concept Grants for Exploratory Research (EAGER) grant for research that shows signs of transforming its domain but remains untested.
In 2012, Mental Canvas had evolved to such a state that it received an Innovation Corps (I-CORPS) grant, a NSF funding mechanism designed to usher work previously funded by public grants into a commercialization of technology. From the I-CORPS program, Dorsey’s research was funneled into the Small Business Innovation Research (SBIR) program that moves scientific discoveries into the private sector.
“Being funded by SBIR means that we could explore what seem to be the most promising ways to commercialize this technology, as opposed to being locked into an investor’s priorities,” Dorsey notes. “There was much greater focus on more rigorous and challenging potential applications that will have a higher impact in the world.”
Back when she first started on the project, the technology required high-end graphics processing units and extensive CAD training to operate. Now, it is available to the ordinary consumer using an everyday tablet like an iPad. This also means researchers are no longer tethered to heavy, expensive computers and displays — they can take their work with them.
This lightweight, mobile approach makes Mental Canvas an appealing and timely technology.
Whether you’re an archaeologist envisioning a reconstruction, or an automotive designer studying a new form, Mental Canvas provides the freedom to explore many possibilities. Interactive storyboarding even provides a sense of what a shot will look like from multiple perspectives, so it's clear how a video or presentation might flow.
Mental Canvas returns to the drawing board and comes back with an elevated and expanded role for the act of sketching. Dorsey's technology reminds us of the creative process's lasting value, and the rewards of patient investments.