Loading...

Weakly Supervised Semantic Segmentation Using Deep Neural Networks

Khairi Atani, Masoud | 2021

757 Viewed
  1. Type of Document: M.Sc. Thesis
  2. Language: Farsi
  3. Document No: 54180 (19)
  4. University: Sharif University of Technology
  5. Department: Computer Engineering
  6. Advisor(s): Kasaei, Shohreh
  7. Abstract:
  8. Semantic segmentation which is the classification of every pixel in an input image is a fundamental task in the fields of computer vision and scene understanding. Applications of semantic segmentation include usage in autonomous vehicles and robotics. Since in this task dense annotation of images in the dataset is needed, recent methods have been proposed to utilize weakly-supervised and semi-supervised learning using data with weak labels and unlabeled data respectively. Because the amount of fully labeled data might not be sufficient in such methods, some papers have proposed to employ depth input data due to its rich geometrical and local information when available. In this research, an adversarial-learning-based method has been proposed to make use of unlabeled images in a semi-supervised setting using the RGB and depth inputs. In this method, instead of using depth input directly, a depth output is predicted and reconstructed along the semantic segmentation map output using the features present in the RGB image. Therefore, at inference time the modality of the proposed network would be RGB-only since there is no need for a depth input to be fed to the network. To employ the geometric affinities in depth modality, a fusion module has been used to further enhance the RGB and depth features using a cross-attention mechanism. For improving the training performance of the network, ideas such as employment of weighted loss functions, confidence map generation, and spectral normalization have been utilized. Extensive experiments of the proposed method on popular indoor semantic segmentation datasets such as SUN RGB-D and NYU-V2 prove that the proposed method outperforms the current semi-supervised similar methods by 2.1 percent in mIoU metric on SUN RGB-D dataset. The proposed method also has the least difference between fully and semi-supervised compared to similar current methods
  9. Keywords:
  10. Semantic Segmentation ; Deep Neural Networks ; Semi-Supervised Learning ; Adversarial Machine Learning

 Digital Object List

 Bookmark

...see more