By Wayne Gillam | UW ECE News
UW ECE and Physics Associate Professor Arka Majumdar and UW ECE postdoctoral scholar Johannes Fröch are part of an international research team helping to make high-quality, color cameras smaller and lighter for mobile platforms, such as next-generation smartphones, drones, and point-of-care medical devices. The team recently developed a miniature camera that uses an innovative hybrid optical system over 100 times smaller than its commercial counterpart. Shown above: a close-up view of the raised camera “bump” on the back of a typical smartphone today. Majumdar and Fröch’s research advance could help to reduce the size of this camera bump or eliminate it entirely, containing smartphone cameras completely within the phone frame and significantly reducing manufacturing costs. Photo by Ryan Hoover | UW ECE
Our smartphone cameras have become a staple of modern life. Each day, millions of photos and videos are captured with smartphones as people snap pictures and record videos of loved ones and memorable moments, such as playing with a favorite pet, appreciating a dramatic city skyline, or enjoying a beautiful sunset. Meanwhile, other mobile devices that incorporate small cameras, such as aerial drones, continue to grow in popularity as people explore looking at the world around us in new and exciting ways.
Because demand for mobile devices is high, and smaller is often considered to be better, there is an ongoing technological race to make the cameras contained within smartphones, drones, and other mobile platforms smaller and lighter than they already are. But the engineering challenges involved in doing this, whether using traditional refractive camera lenses or more advanced technology, grow greater as cameras continue to shrink in size.
UW ECE and Physics Associate Professor Arka Majumdar and UW ECE postdoctoral scholar Johannes Fröch are part of an international research team taking on these challenges and helping to make cameras smaller and lighter for mobile platforms, such as next-generation smartphones, drones and point-of-care medical devices. As described in their recent paper in Science Advances, the team has developed a miniature camera capable of capturing full-color, high quality images, using an innovative hybrid optical system that is over 100 times smaller than its commercial counterpart.
This new camera’s optical system is considered to be a hybrid, because it combines a traditional refractive lens (similar to the kind of lens found in a pair of glasses) with advanced meta-optics — flat, planar surfaces consisting of microscopic structures known as “nanopillars,” which are used to manipulate light and provide a degree of control at a sub-wavelength scale not possible with refractive lenses. This combination helps to resolve chromatic and geometric imperfections in photos and video, which have long been a problem for researchers using meta-optics alone. The camera’s hybrid system also includes an artificial neural network, which uses its computing power to fill-in image gaps that the optical side of the system may not be able to capture. This three-part hybrid framework is allowing researchers such as Majumdar and Fröch to completely reimagine what an optical system could be.
“We are really thinking about optics as more of an information processing device, rather than as an image-forming device,” Majumdar said. “We want to capture enough information from the scene to create an image, rather than try to collect all of it. If we capture enough information, then the computation backend can form an accurate image. That way, we can reduce the size of our device.”
Majumdar and Fröch collaborated on this project with researchers from Tampere University in Tampere, Finland, and the Science and Technology Facilities Council in Harwell, United Kingdom. Fabrication of the meta-optics contained within the camera took place at the Washington Nanofabrication Laboratory on the UW campus.
The billion-dollar bump
UW ECE and Physics Associate Professor Arka Majumdar (left) and UW ECE postdoctoral scholar Johannes Fröch (right)
On the back of most smartphones today is a small, raised bump, where the phone’s camera resides. The reason this bump is there is because the average smartphone camera contains eight or nine refractive lenses that together create the camera’s optical stack. Although these refractive lenses are tiny, when added together, they need more vertical space than what the depth of a smartphone typically allows.
“That bump is actually a billion-dollar problem,” Majumdar said. “It doesn’t matter that much to the end-user, but to create packaging that accounts for the bump, companies have to spend a lot of money, and that cost gets passed on to the consumer.”
To help address the issue of the bump by reducing the size of the camera’s optical stack, Majumdar and his research team built their camera using one refractive lens and a layer of meta-optics only one micron thick. That’s much thinner than the thickness of a grain of sand (62 to 500 microns) or the width of a single human hair (70 to 100 microns).
In addition to engineering a hybrid optical system that is over 100 times smaller than its commercial counterpart, this camera’s aperture, which is directly proportional to how much visual information the device can take in, is over 10 times larger than similar cameras in the marketplace. And not only is the camera the team developed significantly smaller and lighter, with a much larger aperture than what is commercially available, it even captures images that are of superior quality.
“For me, what this research advance demonstrates is that even when the optics are incredibly small, you can still preserve color and image quality,” Majumdar said. “This points to other applications for this device beyond smartphones or drones.”
It is also worth pointing out that as the device makes its way into the marketplace, less glass and plastic material will be needed to build it than its predecessors. When multiplied over millions of smartphones and devices, this could contribute to significantly reducing the environmental impact of technologies using this type of camera.
A wide range of future applications
Majumdar said he is excited about this new camera and applying its design framework to other applications. Those include incorporating the optical system into point-of-care medical devices, such as endoscopes and angioscopes, which use very small cameras to scan inside arteries, veins, and other hard-to-access areas inside the human body. Other applications, such as object detection in cameras used by autonomous vehicles and robotics are on the horizon, as is using the camera for hyperspectral imaging, which reveals aspects of an image undetectable to the human eye alone. Hyperspectral imaging has several possible uses in precision agriculture, for example, thermal imaging of fields to detect when crops need watering, and hand-held (smartphone) food inspection, which could detect whether fruits and vegetables might contain a bit of rot inside a shiny exterior.
With all these things in mind, Majumdar plans to commercialize aspects of this new technology through his startup company Tunoptix, which is an industry leader in broadband meta-optics imaging. He also plans to work with his UW research team along with researchers at Tampere University and the Science and Technology Facilities Council to package all the discrete components of this camera into one, integrated system that can be commercialized. He expects this miniature camera to make its way into the marketplace within three to five years.
“One of the main things I’d like people to know is that even after significant optimization of devices such as smartphones, there is still a lot of room for improvement,” Majumdar said. “Once you add a computational backend, like we did, and co-design the optics with computation, then a lot of things can be done that were not possible before.”
Funding and support for UW researchers participating in this project was provided by the National Science Foundation and the Defense Advanced Research Projects Agency (DARPA) in the U.S. Department of Defense. Learn more about this research by reading “Miniature Color Camera via Flat Hybrid Meta-Optics” in Science Advances or this press release from Tampere University. More information is also available by contacting Arka Majumdar.