Shape2.5D: A Dataset of Texture-less Surfaces for Depth and Normals Estimation

Muhammad Saif Ullah Khan1,2, Sankalp Sinha1,2, Didier Stricker1,2, Marcus Liwicki3, Muhammad Zeshan Afzal1,2
[1] Rhineland-Palatinate Technical University of Kaiserslautern-Landau (RPTU)
[2] German Research Center for Artificial Intelligence (DFKI)
[3] Luleå University of Technology
IEEE Access
Paper Dataset Code arXiv

The Shape2.5D dataset is the first large-scale dataset for depth and normals estimation of texture-less objects. It comprises 48 synthetic objects and six real-world objects. Each object is rendered RGB images from different viewpoints without textures. We provide corresponding depth maps, normal maps, and camera information.

Abstract

Reconstructing texture-less surfaces poses unique challenges in computer vision, primarily due to the lack of specialized datasets that cater to the nuanced needs of depth and normals estimation in the absence of textural information. We introduce "Shape2.5D," a novel, large-scale dataset designed to address this gap. Comprising 1.17 million frames spanning over 39,772 3D models and 48 unique objects, our dataset provides depth and surface normal maps for texture-less object reconstruction. The proposed dataset includes synthetic images rendered with 3D modeling software to simulate various lighting conditions and viewing angles. It also includes a real-world subset comprising 4,672 frames captured with a depth camera. Our comprehensive benchmarks demonstrate the dataset's ability to support the development of algorithms that robustly estimate depth and normals from RGB images and perform voxel reconstruction. Our open-source data generation pipeline allows the dataset to be extended and adapted for future research.

Image Depth Map Normal Map


Dataset

The trained models that were used to evaluate the dataset to report benchmarking results are available for download here (480 MB). You can download this archive and use the eval.sh bash script in the source code repository to reproduce the results reported in the paper yourself.

Alternatively, if you would like to train the networks from scratch, you can use the train.py script in the same repository.

Category Objects Num. Objects Download
animals asian_dargon, bunny, cats, dragon, duck, pig 6 Link (27 GB)
clothing cape, dress, hoodie, jacket, shirt, suit, tracksuit, tshirt 8 Link (37 GB)
furniture armchair, bed, chair, rocking_chair, sofa, table 6 Link (33 GB)
misc diego, kettle, plants, skeleton, teapot 5 Link (29 GB)
statues armadillo, buddha, lucy, roman, thai_statue 5 Link (21 GB)
vehicles bicycle, car, jeep, ship, spaceship 5 Link (18 GB)
shapenet plane, bench, cabinet, car, chair display lamp
speaker rifle sofa table phone watercraft 		13 Link (18 GB)
real chair, tshirt, hoody, lamp, 6 Link (5 GB)

Related Links

These models were obtained from several sources in the public domain.

  1. The Stanford 3D Scanning Repository

    Models obtained from this repository include 5 Stanford models and 2 XYZ RGB models. These include bunny, dragon, buddha, armadillo, lucy, asian_dargon, and thai.

  2. Keenan’s 3D Model Repository

    This repository was published by Keenan Crane of Carnegie Mellon University under the CC0 1.0 Universal (CC0 1.0) Public Domain License. duck, pig, skeleton and diego were obtained from here.

  3. Other Sources

    The teapot is Martin Newell’s Utah Teapot, and the remaining 24 models were all obtained for free from CGTrader with a Royalty Free License. A complete list of sources for each individual model can be found here.

BibTeX

@article{khan2024shape,
  author={Khan, Muhammad Saif Ullah and Sinha, Sankalp and Stricker, Didier and Liwicki, Marcus and Afzal, Muhammad Zeshan},
  journal={IEEE Access}, 
  title={Shape2.5D: A Dataset of Texture-less Surfaces for Depth and Normals Estimation}, 
  year={2024},
  volume={},
  number={},
  pages={1-1},
  doi={10.1109/ACCESS.2024.3492703}
}