GPS-Net: Graph-based Photometric Stereo Network

 

Zhuokun Yao1, Kun Li1*, Ying Fu2, Haofeng Hu1, Boxin Shi3*

1 Tianjin University   2 Beijing Institute of Technology   3 Peking University  

* Corresponding authors

 

Abstract

Learning-based photometric stereo methods predict the surface normal either in a per-pixel or an all-pixel manner. Per-pixel methods explore the inter-image intensity variation of each pixel but ignore features from the intra-image spatial domain. All-pixel methods explore the intra-image intensity variation of each input image but pay less attention to the inter-image lighting variation. In this paper, we present a Graph-based Photometric Stereo Network, which unifies per-pixel and all-pixel processings to explore both inter-image and intra-image information. For per-pixel operation, we propose the Unstructured Feature Extraction Layer to connect an arbitrary number of input image-light pairs into graph structures, and introduce Structure-aware Graph Convolution filters to balance the input data by appropriately weighting shadows and specular highlights. For all-pixel operation, we propose the Normal Regression Network to make efficient use of the intraimage spatial information for predicting a surface normal map with rich details. Experimental results on the real-world benchmark show that our method achieves excellent performance under both sparse and dense lighting distributions.

 

[Code] [Paper]

 

Fig 1. Method overview.

 


Results

 

 

Table 1. Quantitative comparsion on the DiLiGenT benchmark.

 

 

 

Fig 2. Qualitative comparsion on more real-world datasets.

 

 



Citation

Zhuokun Yao, Kun Li, Ying Fu, Haofeng Hu, and Boxin Shi, "GPS-Net: Graph-based Photometric Stereo Network", in Proc. Neural Information Processing Systems (NIPS), 2020.

 

@inproceedings{yao2020gps,
  author = {Zhuokun Yao and Kun Li and Ying Fu and Haofeng Hu and Boxin Shi},
  title = {{GPS-Net}: Graph-based Photometric Stereo Network},
  booktitle={Neural Information Processing Systems (NIPS)},
  year={2020},
}