• Subscribe

Seeing around corners

Speed read
  • New camera system can reconstruct moving objects hidden around a corner
  • Laser 10,000x faster than in previous devices is quick enough to monitor out-of-sight movement
  • Detecting around-the-corner objects may prove invaluable for autonomous cars

David Lindell, a graduate student in electrical engineering at Stanford University, donned a high visibility tracksuit and got to work, stretching, pacing and hopping across an empty room. Through a camera aimed away from Lindell – at what appeared to be a blank wall – his colleagues could watch his every move.

<strong> Warming up.</strong> David Lindell tests a new around-the-corner camera system’s ability to reconstruct his movements in near-real-time. Courtesy David Lindell.That’s because, hidden to the naked eye, he was being scanned by a high powered laser and the single particles of light he reflected onto the walls around him were captured and reconstructed by the camera’s advanced sensors and processing algorithm.

“People talk about building a camera that can see as well as humans for applications such as autonomous cars and robots, but we want to build systems that go well beyond that,” said Gordon Wetzstein, an assistant professor of electrical engineering at Stanford. “We want to see things in 3D, around corners and beyond the visible light spectrum.”

The camera system Lindell tested, which the researchers presented at the SIGGRAPH 2019 conference, builds upon previous around-the-corner cameras this team developed.

The new camera captures more light from a greater variety of surfaces, sees wider and farther away, and is fast enough to monitor out-of-sight movement – such as Lindell’s calisthenics – for the first time. Someday, the researchers hope superhuman vision systems could help autonomous cars and robots operate even more safely than they would with human guidance.

Practicality and seismology

Keeping their system practical is a high priority for these researchers. The hardware they chose, the scanning and image processing speeds, and the style of imaging are already common in autonomous car vision systems.

A new camera system can reconstruct room-size scenes and moving objects that are hidden around a corner out of sight. Courtesy Stanford Computational Imaging Lab.

Previous systems for viewing scenes outside a camera’s line of sight relied on objects that either reflect light evenly or strongly. But real-world objects, including shiny cars, fall outside these categories. That's why this system can handle light bouncing off a range of surfaces, including disco balls, books and intricately textured statues.

Central to their advance was a laser 10,000 times more powerful than what they were using a year ago. The laser scans a wall opposite the scene of interest and that light bounces off the wall, hits the objects in the scene, bounces back to the wall and to the camera sensors.

By the time the laser light reaches the camera only specks remain, but the sensor captures every one, sending it along to a highly efficient algorithm, also developed by this team, that untangles these echoes of light to decipher the hidden tableau.

“When you’re watching the laser scanning it out, you don’t see anything,” described Lindell. “With this hardware, we can basically slow down time and reveal these tracks of light. It almost looks like magic.”

The system can scan at four frames per second. It can reconstruct a scene at speeds of 60 frames per second on a computer with a graphics processing unit, which enhances graphics processing capabilities.

<strong>How to do it.</strong> The imaging system records the time it takes for laser light to scatter off the wall, reflect from the hidden object, and return to the wall. The 3D geometry of the hidden object can be reconstructed from the timing measurements. Courtesy David Lindell.To advance their algorithm, the team looked to other fields for inspiration. The researchers were particularly drawn to seismic imaging systems – which bounce sound waves off underground layers of Earth to learn what’s beneath the surface – and reconfigured their algorithm to likewise interpret bouncing light as waves emanating from the hidden objects. The result was the same high-speed and low-memory usage with improvements in their abilities to see large scenes containing various materials.

“There are many ideas being used in other spaces – seismology, imaging with satellites, synthetic aperture radar – that are applicable to looking around corners,” said Matthew O’Toole, an assistant professor at Carnegie Mellon University who was previously a postdoctoral fellow in Wetzstein’s lab. “We’re trying to take a little bit from these fields and we’ll hopefully be able to give something back to them at some point.”

Humble steps

Being able to see real-time movement from otherwise invisible light bounced around a corner was a thrilling moment for this team but a practical system for autonomous cars or robots will require further enhancements.

<strong>Laser leap.</strong> The camera system’s hardware prototype collects measurements with a laser that is 10,000 times more powerful than used in previous systems. Courtesy David Lindell.“It’s very humble steps. The movement still looks low-resolution and it’s not super-fast but compared to the state-of-the-art last year it is a significant improvement,” said Wetzstein. “We were blown away the first time we saw these results because we’ve captured data that nobody’s seen before.

The team hopes to move toward testing their system on autonomous research cars, while looking into other possible applications, such as medical imaging that can see through tissues. Among other improvements to speed and resolution, they’ll also work at making their system even more versatile to address challenging visual conditions that drivers encounter, such as fog, rain, sandstorms and snow.

Read more:

Read the original article on Stanford's site.

Join the conversation

Do you have story ideas or something to contribute? Let us know!

Copyright © 2019 Science Node ™  |  Privacy Notice  |  Sitemap

Disclaimer: While Science Node ™ does its best to provide complete and up-to-date information, it does not warrant that the information is error-free and disclaims all liability with respect to results from the use of the information.

Republish

We encourage you to republish this article online and in print, it’s free under our creative commons attribution license, but please follow some simple guidelines:
  1. You have to credit our authors.
  2. You have to credit ScienceNode.org — where possible include our logo with a link back to the original article.
  3. You can simply run the first few lines of the article and then add: “Read the full article on ScienceNode.org” containing a link back to the original article.
  4. The easiest way to get the article on your site is to embed the code below.