Reinforcement Learning for Photometric Tuning Improves Ovarian Lesion Segmentation
Article Sidebar
Main Article Content
Abstract
Image segmentation is a fundamental task in computer vision and has become an essential component of many real-world applications, particularly in medical image analysis. In ovarian tumor diagnosis, accurate ultrasound image segmentation can assist clinicians by reducing the time required for lesion delineation and facilitating faster treatment planning. However, ultrasound imaging remains challenging because images typically exhibit low contrast, speckle noise, and poorly defined lesion boundaries. These characteristics substantially degrade the performance of automated segmentation models, and reduce the accuracy of tumor localization. In this study, we introduce a reinforcement learning (RL)based method designed to improve image contrast, and effectively mitigate this problem. We apply a Proximal Policy Optimization (PPO) Agent that applies small, structure-preserving photometric transformations (contrast/gamma adjustment, noise reduction and contrast-limited adaptive histogram equalization) to the input image and a pretrained model (U-Net) to evaluate the output results. The Agent receives a distinguishable reward based on the improvement in the soft-Dice Similarity Coefficient (S-DSC) from one step to the next, with penalties for unnecessary actions and an explicit stopping action. This method improves S-DSC over the baseline U-Net without retraining or architectural changes to the segmentation set. The proposed method consistently improved segmentation performance across all evaluated cases. The most substantial improvement was observed in challenging ultrasound images, where the Dice score increased by up to 78%. These results demonstrate that adaptive, learned photometric adjustments can effectively enhance inference-time segmentation without requiring retraining or architectural modifications to the segmentation model. The approach is designed to be architecture-independent, easy to integrate with frozen segmenters, and to maintain full interpretability across step-by-step action traces.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
H. Sung, J. Ferlay, R. L. Siegel, M. Laversanne, I. Soerjomataram, A. Jemal and F. Bray, “Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries,” CA: A Cancer Journal for Clinicians, vol. 71, no. 3, pp. 209–249, 2021.
O. Ronneberger, P. Fischer and T. Brox, “UNet: Convolutional networks for biomedical image segmentation,” in International Conference on Medical Image Computing and ComputerAssisted Intervention (MICCAI), LNCS, vol. 9351, Springer, pp. 234–241, 2015.
W. Jiangtao, N. I. R. Ruhaiyem and F. Panpan, “A comprehensive review of U-Net and its variants: Advances and applications in medical image segmentation,” IET Image Processing, vol. 19, no. 1, p. e70019, 2025.
N. Hung, T. T. Huong, N. Tan,T. Doan and N. Nam-Hoang, “Machine learning applications for delivery time prediction and freight planning,” Informatics and Automation, vol. 24, no. 5, pp. 1379–1407, 2025.
N. Moshkov, L. Bonte, N. Vandewiele, J. M. S. S´anchez, W. Alkema, J. Dambre and F. Wyffels, “Test-time augmentation for deep learningbased cell segmentation,” Scientific Reports, vol. 10, no. 1, p. 5068, 2020.
P. Conde, C. Premebida and P. Coimbra, “Adaptive-TTA: Accuracy-consistent weighted test time augmentation method for the uncertainty calibration of deep learning classifiers,” in Proceedings of the British Machine Vision Conference (BMVC), p. 869, 2022.
Y. Sun, X. Wang, Z. Liu, J. Miller, A. A. Efros and M. Hardt, “Test-time training with selfsupervision for generalization under distribution shifts,” in Proceedings of the International Conference on Machine Learning (ICML), PMLR, vol. 119, pp. 9229–9248, 2020.
N. Karani, E. Erdil, K. Chaitanya and E. Konukoglu, “Test-time adaptable neural networks for robust medical image segmentation,” Medical Image Analysis, vol. 68, p. 101907, Feb. 2021. [Online]. Available: http://dx.doi.org/ 10.1016/j.media.2020.101907
R. J. G. van Sloun, R. Cohen and Y. C. Eldar, “Deep Learning in Ultrasound Imaging,” in Proceedings of the IEEE, vol. 108, no. 1, pp. 11-29, Jan. 2020.
M. Hu, J. Zhang, L. Matkovic, T. Liu and X. Yang, “Reinforcement learning in medical image analysis: Concepts, applications, challenges, and future directions,” arXiv preprint arXiv:2206.14302, 2022.
K. Zuiderveld, “Contrast limited adaptive histogram equalization,” in Graphics Gems IV, P. S. Heckbert, Ed. Academic Press, pp. 474–485, 1994.
Q. Zhao, S. Lyu, W. Bai, L. Cai, B. Liu, G. Cheng, M. Wu, X. Sang, M. Yang and L. Chen, “MMOTU: A multi-modality ovarian tumor ultrasound image dataset for unsupervised crossdomain semantic segmentation,” arXiv preprint arXiv:2207.06799, 2022.
Y. Yu and S. T. Acton, “Speckle reducing anisotropic diffusion,” in IEEE Transactions on Image Processing, vol. 11, no. 11, pp. 1260-1270, Nov. 2002.
K. He, X. Zhang, S. Ren and J. Sun, “Deep Residual Learning for Image Recognition,” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, pp. 770-778, 2016.
P. Reangsuntea, A. Reangsuntea, P. Boonsrimuang, K. Mori and H. Kobayashi, “Iterative Based Time Domain Equalization Method for DFTS-OFD in Highly Mobile Environments,” ECTI-CIT Transactions, vol. 11, no. 1, pp. 28–39, Jul. 2017.
E. D. Cubuk, B. Zoph, D. Man´e, V. Vasudevan and Q. V. Le, “AutoAugment: Learning Augmentation Strategies From Data,” 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, pp. 113-123, 2019. 537
D. Wang, E. Shelhamer, S. Liu, B. Olshausen and T. Darrell, “Tent: Fully test-time adaptation by entropy minimization,” arXiv preprint arXiv:2006.10726, 2021.
F.-C. Ghesu, B. Georgescu, T. Zach, Y. Zheng, J. Petersen and D. Comaniciu, “An artificial agent for anatomical landmark detection in medical images,” in International Conference on Medical Image Computing and ComputerAssisted Intervention (MICCAI) pp. 229–237, 2017.
J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017.
A. Dosovitskiy et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” in Proceedings of International Conference on Learning Representations, 2021.
Z. Zhou, M. M. R. Siddiquee, N. Tajbakhsh and J. Liang, “Unet++: A nested u-net architecture for medical image segmentation,” in Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, vol. 11045, pp. 3–11, 2018.
T. -Y. Lin, P. Doll´ar, R. Girshick, K. He, B. Hariharan and S. Belongie, “Feature Pyramid Networks for Object Detection,” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, pp. 936–944, 2017.
G. -V. Nguyen and T. Huynh-The, “Enhancing Aerial Semantic Segmentation With Feature Aggregation Network for DeepLabV3+,” in IEEE Geoscience and Remote Sensing Letters, vol. 21, pp. 1-5, Art no. 2504205, 2024.
L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff and H. Adam, “Encoder-decoder with atrous separable convolution for semantic image segmentation,” in Computer Vision – ECCV 2018, vol. 11211, pp. 833–851, 2018.
H. Zhao, J. Shi, X. Qi, X. Wang and J. Jia, “Pyramid Scene Parsing Network,” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, pp. 6230-6239, 2017.
