Method for Semantic Image Segmentation Based on the Neural Network with Gabor Filters

Cover Page

Cite item

Full Text

Open Access Open Access
Restricted Access Access granted
Restricted Access Subscription or Fee Access

Abstract

The article is devoted to the use of Gabor filters to improve the efficiency of convolutional neural networks (CNN) in image analysis tasks, in particular, segmentation. The application of Gabor filters as an adaptive component in the initial layers of CNN is considered, which allows improving the extraction of texture and structural features. To achieve an optimal balance between the number of trainable parameters and accuracy, adaptive Gabor filters are proposed, which increase the number of input channels without significantly complicating the model. A comparative analysis of architectures using PSPNet for image segmentation modified with adaptive Gabor filters is carried out. Limitations on the size of filters that ensure acceptable computational costs are considered. The relevance of the approach on a dataset for image segmentation is confirmed, demonstrating an improvement in accuracy with a minimal increase in the number of parameters.

Full Text

Restricted Access

About the authors

E. A. Murin

Faculty of Computational Mathematics and Cybernetics, Moscow State University

Email: kryl@cs.msu.ru

Laboratory of Mathematical Methods of Image Processing

Russian Federation, Moscow, 119991

D. V. Sorokin

Faculty of Computational Mathematics and Cybernetics, Moscow State University

Email: dsorokin@cs.msu.ru

Laboratory of Mathematical Methods of Image Processing

Russian Federation, Moscow, 119991

A. S. Krylov

Faculty of Computational Mathematics and Cybernetics, Moscow State University

Author for correspondence.
Email: kryl@cs.msu.ru

Laboratory of Mathematical Methods of Image Processing

Russian Federation, Moscow, 119991

References

  1. Fogel Itzhak, Sagi Dov. Gabor filters as texture discriminator // Biological cybernetics. 1989. Vol. 61. No. 2. P. 103–113.
  2. Coding facial expressions with Gabor wavelets / Michael Lyons, Shigeru Akamatsu, Miyuki Kamachi, Jiro Gyoba // Proceedings Third IEEE international conference on automatic face and gesture recognition // IEEE. 1998. P. 200–205.
  3. Kwolek Bogdan. Face detection using convolutional neural networks and Gabor filters // Artificial Neural Networks: Biological Inspirations – ICANN 2005: 15th International Conference, Warsaw, Poland, September 11–15, 2005. Proceedings, Part I 15 / Springer. – 2005. P. 551–556.
  4. Calderon Andres, Roa Sergio, Victorino Jorge. Handwritten digit recognition using convolutional neural networks and Gabor filters // Proc. Int. Congr. Comput. Intell. 2003. P. 1–9.
  5. Chang Shuo-Yiin, Morgan Nelson. Robust CNN-based speech recognition with Gabor filter kernels // Fifteenth annual conference of the international speech communication association. 2014. P. 905–909.
  6. Sarwar Syed Shakib, Panda Priyadarshini, Roy Kaushik. Gabor filter assisted energy efficient fast learning convolutional neural net-works // 2017 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED) / IEEE. 2017. P. 1–6.
  7. Gabor convolutional networks / Shangzhen Lu-an, Chen Chen, Baochang Zhang et al. // IEEE Transactions on Image Processing. 2018. Vol. 27. No. 9. P. 4357–4366.
  8. Adaptive Gabor convolutional networks / Ye Yuan, Li-Na Wang, Guoqiang Zhong et al. // Pattern Recognition. 2022. Vol. 124. P. 108495.
  9. Learnable Gabor Convolutional Networks / Guoqiang Zhong, Wei Gao, Wencong Jiao et al. // International Conference on Neural Information Processing / Springer. 2019. P. 305–313.
  10. Residual Gabor convolutional network and FV-Mix exponential level data augmentation strategy for finger vein recognition / Yifan Wang, Huimin Lu, Xiwen Qin, Jianwei Guo // Expert Systems with Applications. 2023. Vol. 223. P. 119874.
  11. Learning Gabor texture features for fine-grained recognition / Lanyun Zhu, Tianrun Chen, Jianxiong Yin et al. // Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023. P. 1621–1631.
  12. Osa Priscilla Indira, Zerubia Josiane, Kato Zoltan. AYANet: A Gabor Wavelet-Based and CNN-Based Double Encoder for Building Change Detection in Remote Sensing // Pattern Recognition / Ed. by Apostolos Antonacopoulos, Subhasis Chaudhuri, Rama Chellappa et al. Cham: Springer Nature Switzerland, 2025. P. 131–146.
  13. Gabor D. Theory of communication: Journal of the Institute of Electrical Engineers // London. 1946. Vol. 93. P. 429–457.
  14. Krizhevsky Alex, Sutskever Ilya, Hinton Geof-frey E. Imagenet classification with deep convolutional neural networks // Communications of the ACM. 2017. Vol. 60. No. 6. P. 84–90.
  15. Fast and efficient facial expression recognition using a Gabor convolutional network / Ping Jiang, Bo Wan, Quan Wang, Jiang Wu // IEEE Signal Processing Letters. 2020. Vol. 27. P. 1954–1958.
  16. Мурин Е.А., Сорокин Д.В., Крылов А.С. Методы классификации гистологических изображений на основе сверточных нейронных сетей с фильтром Габора // GRAPHICON 2024. Материалы 34-й Международной конференции по компьютерной графике и машинному зрению. Омск: Омский государственный технический университет, 2024. С. 656–668.
  17. Deep residual learning for image recognition / Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun // Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. P. 770–778.
  18. Pyramid scene parsing network / Heng-shuang Zhao, Jianping Shi, Xiaojuan Qi et al. // Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. P. 2881–2890.
  19. Gradient-based learning applied to document recognition / Yann LeCun, L´eon Bottou, Yoshua Bengio, Patrick Haffner // Proceedings of the IEEE. 1998. Vol. 86. No. 11. P. 2278–2324.
  20. The cityscapes dataset for semantic urban scene understanding / Marius Cordts, Mohamed Omran, Sebastian Ramos et al. // Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. P. 3213–3223.

Supplementary files

Supplementary Files
Action
1. JATS XML
Download
2. Fig. 1. Block of 16 real parts of Gabor filters, oriented at angles with a step of 11.25°.

Download (864KB)
Indexing metadata
3. Fig. 2. AlexNet filters trained on the ImageNet dataset.

Download (613KB)
Indexing metadata
4. Fig. 3. Schematic of a convolutional layer with a Gabor filter.

Download (297KB)
Indexing metadata
5. Fig. 4. General view of the PSPNet architecture.

Download (960KB)
Indexing metadata
6. Fig. 5. ResNet block (a) and GResNet (b)

Download (585KB)
Indexing metadata
7. Fig. 6. Network architecture.

Download (420KB)
Indexing metadata
8. Fig. 7. Examples of Gabor filters obtained during the training process: FL-G (a), iAGCN (b), AGCN (c).

Download (3MB)
Indexing metadata

Copyright (c) 2025 Russian Academy of Sciences