Machine Vision with Single Pixel Camera and Deep Learning based Signal Processing

Its defining features are that it works without areal optical sensors by leveraging methods from Compressed Sensing, that a Neural Network performs the data processing and that it directly extracts features like size and positioning of objects in a 3D space. The setup offers multiple advantages: It reduces hardware requirements for the data acquisition by simplifying the sensor and greatly lightens the processing side by directly returning relevant scene information, eliminating explicit image reconstruction. Thus, this approach belongs to the still sparsely covered area of image free methods.

We realize this based on a Single Pixel camera setup, which works by projecting structured dark-light-patterns onto a scene. A photodiode collects the diffusely reflected light, yielding a single intensity value. This intensity depends on the structure of both the masks and the objects in the scene, so by using multiple different masks, we encode the objects’ characteristics. Using appropriate data processing, in our case a specially trained Neural Net, we are able to reconstruct object parameters of interest directly.