Test pattern

Monochrome

image

A poster of the test pattern imaged by a   monochrome

camera

Processor-based filtering (Matlab): edge-detection with a derivative operator

Optical filtering: edge detection with  a derivative operator

Image

Filtering

Scene-space Filtering

Related Publications

Y. Jean. Optical Signal Processing with Illumination-encoded Filters. Journal of Computer Vision and Image Understanding 115 (2011), pp. 561-575

Y. Jean, “Local Optical Operators for Subpixel Scene Analysis”, International Conference Pattern Recognition ‘10.

Y. Jean, “Orthographic projection for optical signal processing”, European Conference Computer Vision’ 08: Omnidirectional Vision, Camera Networks, and Non-classical Cameras, October ‘08

Y. Jean, “Scene-space feature detectors”, IEEE Conference Computer Vision Pattern Recognition ‘07: Beyond Multiview Geometry, 2007

Orthographic Lightfield Projector Design: digital projector + Fresnel lens + calibrated parameters

A calibration step

Orthographic Lightfield Device

Complete device including monochrome camera for optical integration

Computer vision techniques rely on computational tools from the signal processing framework. These tools are applied to images of a scene to infer the scene contents. Since an image is a degraded representation of the rich 3D scene it would make more sense to apply such computation to the scene itself.

    In this project we define Scene-space signal processing as the formulation of signal processing computations directly in the scene. Computations normally performed on a computer processor are represented optically with coded illumination, leveraging the native bidirectional reflectance distribution functions (BRDF) in the scene, and integrating the results with camera pixels (basically, capture a camera image). Scalar functions are substituted with intensity-coded projector illumination patterns and, assuming Lambertian BRDF models, products and sum terms derived from reflection (multiplication) and pixel exposure (integration).

    To illustrate these ideas it is useful to compare an image processing example against Scene-space processing a simple related object, such as a print or poster. In the figure above a Gaussian derivative convolution filter is applied to the test pattern to extract edges using image processing on a computer processor on the left column, and the same filter is applied to a print of the test pattern with our Scene-space filtering algorithm on the right. In both cases arbitrary thresholds were used. Note that the results are comparable in recovering edge features.

    In addition to developing a Scene-space signal processing algorithm we also presented an orthographic lightfield projector design and calibration techniques to align the projection’s optical sampling geometry with the usual regular geometry of sampling theory.