Photometric Stereo: Keep the Camera Still and Move the Light

A maker wants to show the shallow lettering on a handmade tile. In one photograph the lettering almost disappears. With the lamp on the other side, the same letters stand out, but a new shadow hides a corner. Which photograph tells the truth about the tile? Both record something useful. Neither, on its own, separates the shape of the surface from the way it was lit.

Photometric stereo uses that change in illumination as evidence. Keep the object and camera still, photograph the scene under different lights, and compare what happens at the same points in the image. A learning system can use those observations to estimate which way each visible patch faces. Before using the output in a product preview or an archive, check whether those estimates describe the surface or reflect an assumption about the light.

An estimate of direction answers a different question from depth

In his 1980 paper introducing photometric stereo, Robert J. Woodham described varying the direction of the incoming light while keeping the viewpoint fixed. With the scene held still, a pixel can refer to the same visible patch in successive images. Changes in its recorded brightness then provide clues about its orientation, subject to assumptions about illumination and reflectance.

Imagine a tiny arrow sticking straight out from the tile. That arrow represents the surface normal at that spot. On the flat face, the arrows point in much the same direction. At the edge of a raised letter, they turn with the slope. A normal map records estimates of those directions across the image. It does not directly report how far each point is from the camera, and a single view does not reveal the back of the tile.

That distinction is why the method differs from two-camera stereo depth. The latter looks for where the same feature appears in different views and uses that shift to estimate distance. Photometric stereo keeps the view fixed and looks for brightness changes. The two methods can support different parts of a project, but success at one is no substitute for checking the other.

What a learning system changes

A simple reflectance model can be a useful starting point for a matte object. A glossy glaze presents a different problem: a small area can become bright when a highlight falls on it, even though the surface has not moved. A shadow cast by one raised letter can also darken another part of the tile. The model must interpret the changes across images rather than treat a bright or dark pixel as a direct reading of its slope.

In PS-FCN, presented at ECCV in 2018, Guanying Chen, Kai Han and Kwan-Yee K. Wong used a neural network to estimate normal maps from sets of images. Their calibrated method takes the associated light directions and can process sets of different sizes without requiring the photographs to arrive in a prescribed order. The authors trained on synthetic data and reported results on real benchmark images. That is evidence about the method in those experiments. It does not tell a maker whether an untested capture of a particular glazed tile will preserve the shape of its lettering.

Test the surface you intend to describe

Begin with a fixed camera and a stationary object. Keep the framing and focus consistent, and record the illumination used for each photograph. If your method expects known light directions, follow its calibration procedure. A sequence from a handheld camera that also changes its exposure can be attractive for a listing but makes it harder to isolate what the changing lamp tells you.

• Start with a shape you can check. A matte object with two flat faces meeting at a known angle gives you a way to compare the estimated directions. Examine the middle of the faces before looking at the joint.
• Then test the difficult material. Add a glossy patch, a recessed feature or another detail that matters to the real job. Check the normal map alongside the original photographs, especially where a highlight clips or a shadow obscures part of the feature.
• Inspect the result under a new virtual light if your workflow supports it. A relit preview can expose a ridge that looks inverted or a boundary that has gone soft. Treat that as a diagnostic, and compare the suspect detail with the physical object or a known reference.

If you try a second capture with more light positions, check whether it adds evidence where you need it. Another photograph with the same part in shadow is less useful for that part than a view from the same camera with illumination that reaches it. Keep a record of the light positions so a colleague can repeat the comparison instead of guessing which change made the difference.

Match the claim to the evidence

For an online preview, the aim might be to show how the lettering catches the light. For documenting the height of that lettering, you need a suitable measurement and an independent check. Those uses set different standards. Decide in advance which details a viewer needs to see and which dimensions a customer might rely on. Save the original photographs and note which features remain uncertain. The most useful normal map is one whose limits you can explain when somebody asks what, exactly, it shows.

Sources

• Robert J. Woodham, Photometric Method for Determining Surface Orientation from Multiple Images (1980).
• Guanying Chen, Kai Han and Kwan-Yee K. Wong, PS-FCN: A Flexible Learning Framework for Photometric Stereo (ECCV, 2018).