New holographic camera sees the unseen with high precision

• Link to: Northwestern Now Story

• Technology combines two light waves from lasers to create a synthetic wave field
• The synthetic wave field reflected by a hidden object is captured as a hologram
• An algorithm reconstructs the hologram to reveal an image of the obscured object
• The camera could be used to help drivers see around corners to avoid accidents, as a noninvasive medical imaging device, for industrial inspection and more

EVANSTON, Ill. — Northwestern University researchers have invented a new high-resolution camera that can see the unseen — including around corners and through scattering media, such as skin, fog or potentially even the human skull.

Called synthetic wavelength holography, the new method works by indirectly scattering coherent light onto hidden objects, which then scatters again and travels back to a camera. From there, an algorithm reconstructs the scattered light signal to reveal the hidden objects. Due to its high temporal resolution, the method also has potential to image fast-moving objects, such as the beating heart through the chest or speeding cars around a street corner.

The study was published today (Nov. 17) in the journal Nature Communications.

The relatively new research field of imaging objects behind occlusions or scattering media is called non-line-of-sight (NLoS) imaging. Compared to related NLoS imaging technologies, the Northwestern method can rapidly capture full-field images of large areas with submillimeter precision. With this level of resolution, the computational camera could potentially image through the skin to see even the tiniest capillaries at work.

While the method has obvious potential for noninvasive medical imaging, early-warning navigation systems for automobiles and industrial inspection in tightly confined spaces, the researchers believe potential applications are endless. 

“Our technology will usher in a new wave of imaging capabilities,” said Northwestern’s Florian Willomitzer, first author of the study. “Our current sensor prototypes use visible or infrared light, but the principle is universal and could be extended to other wavelengths. For example, the same method could be applied to radio waves for space exploration or underwater acoustic imaging. It can be applied to many areas, and we have only scratched the surface.”

Willomitzer is a research assistant professor of electrical and computer engineering at Northwestern’s McCormick School of Engineering. Northwestern co-authors include Oliver Cossairt, associate professor of computer science and electrical and computer engineering, and former Ph.D. student Fengqiang Li. The Northwestern researchers collaborated closely with Prasanna Rangarajan, Muralidhar Balaji and Marc Christensen, all researchers at Southern Methodist University.

Intercepting scattered light 

Seeing around a corner versus imaging an organ inside the human body might seem like very different challenges, but Willomitzer said they are actually closely related. Both deal with scattering media, in which light hits an object and scatters in a manner that a direct image of the object can no longer be seen.

“If you have ever tried to shine a flashlight through your hand, then you have experienced this phenomenon,” Willomitzer said. “You see a bright spot on the other side of your hand, but, theoretically, there should be a shadow cast by your bones, revealing the bones’ structure. Instead, the light that passes the bones gets scattered within the tissue in all directions, completely blurring out the shadow image.”

The goal, then, is to intercept the scattered light in order to reconstruct the inherent information about its time of travel to reveal the hidden object. But that presents its own challenge.

“Nothing is faster than the speed of light, so if you want to measure light’s time of travel with high precision, then you need extremely fast detectors,” Willomitzer said. “Such detectors can be terribly expensive.” 

Tailored waves

To eliminate the need for fast detectors, Willomitzer and his colleagues merged light waves from two lasers in order to generate a synthetic light wave that can be specifically tailored to holographic imaging in different scattering scenarios.

“If you can capture the entire light field of an object in a hologram, then you can reconstruct the object’s three-dimensional shape in its entirety,” Willomitzer explained. “We do this holographic imaging around a corner or through scatterers — with synthetic waves instead of normal light waves.”

Over the years, there have been many NLoS imaging attempts to recover images of hidden objects. But these methods typically have one or more problems. They either have low resolution, an extremely small angular field of regard, require a time-consuming raster scan or need large probing areas to measure the scattered light signal.

The new technology, however, overcomes these issues and is the first method for imaging around corners and through scattering media that combines high spatial resolution, high temporal resolution, a small probing area and a large angular field of view. This means that the camera can image tiny features in tightly confined spaces as well as hidden objects in large areas with high resolution — even when the objects are moving.

Turning ‘walls into mirrors’ 

Because light only travels on straight paths, an opaque barrier (such as a wall, shrub or automobile) must be present in order for the new device to see around corners. The light is emitted from the sensor unit (which could be mounted on top of a car), bounces off the barrier, then hits the object around the corner. The light then bounces back to the barrier and ultimately back into the detector of the sensor unit.

“It’s like we can plant a virtual computational camera on every remote surface to see the world from the surface’s perspective,” Willomitzer said.

For people driving roads curving through a mountain pass or snaking through a rural forest, this method could prevent accidents by revealing other cars or deer just out of sight around the bend. “This technique turns walls into mirrors,” Willomitzer said. “It gets better as the technique also can work at night and in foggy weather conditions.”

In this manner, the high-resolution technology also could replace (or supplement) endoscopes for medical and industrial imaging. Instead of needing a flexible camera, capable of turning corners and twisting through tight spaces — for a colonoscopy, for example — synthetic wavelength holography could use light to see around the many folds inside the intestines. 

Similarly, synthetic wavelength holography could image inside industrial equipment while it is still running — a feat that is impossible for current endoscopes.

“If you have a running turbine and want to inspect defects inside, you would typically use an endoscope,” Willomitzer said. “But some defects only show up when the device is in motion. You cannot use an endoscope and look inside the turbine from the front while it is running. Our sensor can look inside a running turbine to detect structures that are smaller than one millimeter.”

Although the technology is currently a prototype, Willomitzer believes it will eventually be used to help drivers avoid accidents. “It’s still a long way to go before we see these kinds of imagers built in cars or approved for medical applications,” he said. “Maybe 10 years or even more, but it will come.”

The study, “Fast Non-Line-of-Sight Imaging with High-Resolution and Wide Field of View using Synthetic Wavelength Holography,” was supported by DARPA (REVEAL project HR0011-16-C-0028), the National Science Foundation (CAREER IIS-1453192) and the Office of Naval Research (N00014-15-1-2735).