Single Image Denoising through Downsampling and Self-Resolution Restoration Learning
Article Sidebar
Main Article Content
Abstract
Image denoising using supervised learning effectively removes image noise by learning from available data. However, it may lack efficiency when faced with insufficient data, such as in the case of single images or blind noise. This challenge has led to the adoption of unsupervised learning methods, which utilize the inherent properties of noise to extract and enhance image features. This research aims to leverage the benefits of the downsampling effect for noise removal, even though downsampling may impact image features. Therefore, deep learning must be used to restore image details lost during downsampling. This research proposes the Noisy Low-Resolution to Noisy Super-Resolution (NLR2NSR) framework, which leverages image downsampling to simultaneously reduce image and noise features. A super-resolution network is then used to restore the image features. Experimental results show that under conditions where noise features are less prominent than image features, the NLR2NSR can effectively remove noise and preserve image features using only noisy data for training. However, the NLR2NSR has limitations in handling high-level noise.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
L. Fan, F. Zhang, H. Fan, and C. Zhang, “Brief review of image denoising techniques,” Visual Computing for Industry, Biomedicine, and Art, vol. 2, no. 1, p. 7, 2019.
B. Goyal, A. Dogra, S. Agrawal, B. Sohi, and A. Sharma, “Image denoising review: From classical to state-of-the-art approaches,” Information Fusion, vol. 55, pp. 220–244, 2020.
H. Chen, X. He, L. Qing, Y. Wu, C. Ren, R. E. Sheriff, and C. Zhu, “Real-world single image super-resolution: A brief review,” Information Fusion, vol. 79, pp. 124–145, 2022.
Y. Zhou, S. Song, and N.-M. Cheung, “On classification of distorted images with deep convolutional neural networks,” in 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1213–1217, 2017.
M. Koziarski and B. Cyganek, “Image recognition with deep neural networks in presence of noise dealing with and taking advantage of distortions,” Integrated Computer-Aided Engineering, vol. 24, pp. 337–349, 2017.
W.-C. Siu and K.-W. Hung, “Review of image interpolation and superresolution,” in Proceedings of the 2012 Asia Pacific Signal and Information Processing Association Annual Summit and Conference, 2012, pp. 1–10.
X. Wang, J. Yi, J. Guo, Y. Song, J. Lyu, J. Xu, W. Yan, J. Zhao, Q. Cai, and H. Min, “A review of image super-resolution approaches based on deep learning and applications in remote sensing,” Remote Sensing, vol. 14, p. 5423, 2022.
D. Zheng, S. H. Tan, X. Zhang, Z. Shi, K. Ma, and C. Bao, “An unsupervised deep learning approach for real-world image denoising,” in International Conference on Learning Representations, 2021.
X. Wang, Y. Yang, Z. Guo, Z. Zhou, Y. Liu, Q. Pang, and S. Du, “Real-World Image Super Resolution via Unsupervised Bi-directional Cycle Domain Transfer Learning based Generative Adversarial Network,” ArXiv, vol. abs/2211.10563, 2022.
Y. Quan, M. Chen, T. Pang, and H. Ji, “Self2self with dropout: Learning self-supervised denoising from single image,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1887–1895, 2020.
J. Batson and L. Royer, “Noise2Self: Blind denoising by self-supervision,” in Proceedings of the 36th International Conference on Machine Learning, vol. 97. PMLR, , pp. 524–533, 2019.
J.-C. Yoo and C. W. Ahn, “Image restoration by blind-wiener filter,” IET Image Processing, vol. 8, no. 12, pp. 815–823, 2014.
H. Chen, Z. Cen, C. Wang, S. Lan, and X. Li, “Image restoration via improved wiener filter applied to optical sparse aperture systems,” Optik, vol. 147, pp. 350–359, 2017.
S. Trambadia and P. Dholakia, “Design and analysis of an image restoration using wiener filter with a quality-based hybrid algorithms,” in 2015 2nd International Conference on Electronics and Communication Systems (ICECS), pp. 1318–1323, 2015.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778, 2016.
B. Goyal, A. Dogra, S. Agrawal, B. Sohi, and A. Sharma, “Image denoising review: From classical to state-of-the-art approaches,” Information Fusion, vol. 55, pp. 220–244, 2020.
N. You, L. Han, D. Zhu, and W. Song, “Research on image denoising in edge detection based on wavelet transform,” Applied Sciences, vol. 13, no. 3, p. 1837, 2023.
K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, “Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising,” IEEE Transactions on Image Processing, vol. 26, no. 7, pp. 3142–3155, 2017.
K. Bajaj, D. K. Singh, and M. A. Ansari, “Autoencoders based deep learner for image denoising,” Procedia Computer Science, vol. 171, pp. 1535–1541, 2020.
K. Zhang, W. Zuo, and L. Zhang, “FFDNet: Toward a fast and flexible solution for CNN-based image denoising,” IEEE Transactions on Image Processing, vol. 27, no. 9, pp. 4608–4622, 2018.
H. Zhao, O. Gallo, I. Frosio, and J. Kautz, “Loss functions for image restoration with neural networks,” IEEE Transactions on Computational Imaging, vol. 3, no. 1, pp. 47–57, 2017.
C. Shorten and T. Khoshgoftaar, “A survey on image data augmentation for deep learning,” Journal of Big Data, vol. 6, no. 60, 2019.
M. Xu, S. Yoon, A. Fuentes, and D. S. Park, “A comprehensive survey of image augmentation techniques for deep learning,” Pattern Recognition, vol. 137, p. 109347, 2023.
B. Wu, Z. Liu, Z. Yuan, G. Sun, and C. Wu, “Reducing overfitting in deep convolutional neural networks using redundancy regularizer,” in Artificial Neural Networks and Machine Learning – ICANN 2017, pp. 49–55, 2017.
M. M. Bejani and M. Ghatee, “A systematic review on overfitting control in shallow and deep neural networks,” Artificial Intelligence Review, vol. 54, pp. 6391–6438, 2021.
P. Yang, J. Sturtz, and L. Qingge, “Progress in blind image quality assessment: A brief review,” Mathematics, vol. 11, no. 12, 2023.
M. A. N. I. Fahim, N. Saqib, S. K. Siam, and H. Y. Jung, “Denoising single images by feature ensemble revisited,” Sensors, vol. 22, no. 18, 2022.
A. Krull, T.-O. Buchholz, and F. Jug, “Noise2void - learning denoising from single noisy images,” in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2124–2132, 2019.
Y. Quan, Y. Chen, Y. Shao, H. Teng, Y. Xu, and H. Ji, “Image denoising using complex-valued deep cnn,” Pattern Recognition, vol. 111, p. 107639, 2021.
T. Pang, H. Zheng, Y. Quan, and H. Ji, “Recorrupted-to-recorrupted: Unsupervised deep learning for image denoising,” in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2043–2052, 2021.
J. Lehtinen, J. Munkberg, J. Hasselgren, S. Laine, T. Karras, M. Aittala, and T. Aila, “Noise2noise: Learning image restoration without clean data,” CoRR, vol. abs/1803.04189, 2018.
Y. Kim, H. Ryu, S. Lee, and Y. J. Lee, “Joint demosaicing and denoising based on interchannel nonlocal mean weighted moving least squares method,” Sensors, vol. 20, no. 17, 2020.
W. Xing and K. Egiazarian, “End-to-end learning for joint image demosaicing, denoising and super-resolution,” in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3506– 3515, 2021.
G. Qian, Y. Wang, J. Gu, C. Dong, W. Heidrich, B. Ghanem, and J. S. Ren, “Rethinking learning-based demosaicing, denoising, and super-resolution pipeline,” in 2022 IEEE International Conference on Computational Photography (ICCP), pp. 1–12, 2022.
F. Wang and Y. Li, “Robust kernel estimation for single image blind deconvolution,” in Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012), pp. 481–484, 2012.
P. Sodhani, A. Bordia, and K. Karthik, “Blind content independent noise estimation for multimedia applications,” Procedia Computer Science, vol. 54, pp. 549–557, 2015.
Y. Li, C. Liu, X. You, and J. Liu, “A single-image noise estimation algorithm based on pixel-level low-rank low-texture patch and principal component analysis,” Sensors, vol. 22, no. 22, 2022.
A. Shocher, N. Cohen and M. Irani, “Zero-Shot Super-Resolution Using Deep Internal Learning,” 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, pp. 3118-3126, 2018.
W. Burger and M. J. Burge, “Scale-Invariant Feature Transform (SIFT),” London: Springer London, pp. 609–664, 2016.
B. Brummer and C. De Vleeschouwer, “Natural image noise dataset,” in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1777–1784, 2019.
A. Abdelhamed, S. Lin and M. S. Brown, “A High-Quality Denoising Dataset for Smartphone Cameras,” 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, pp. 1692-1700, 2018.
A. A. Yahya, J. Tan, B. Su, M. Hu, Y. Wang, K. Liu, and A. N. Hadi, “BM3D image denoising algorithm based on an adaptive filtering,” Multimedia Tools and Applications, vol. 79, no. 27, pp. 20391–20427, 2020.
L. Asavaron, D. Rakkrit, and J. Jermphiphut, “A self-validation noise2noise training framework for image denoising,” The Imaging Science Journal, vol. 72, no. 7, pp. 1–16, 2023.
D. Martin, C. Fowlkes, D. Tal and J. Malik, “A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics,” Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, Vancouver, BC, Canada, vol.2, pp. 416-423, 2001.