Multi-source feature fusion network for grape berry instance segmentation

Authors

  • Mingcheng Yao 1. College of Information Science and Engineering, Shandong Agricultural University, Tai’an 271018, Shandong, China
  • Xiaoxia Yang 1. College of Information Science and Engineering, Shandong Agricultural University, Tai’an 271018, Shandong, China
  • Chengming Zhang 1. College of Information Science and Engineering, Shandong Agricultural University, Tai’an 271018, Shandong, China
  • Menxin Wu 2. The National Meteorological Center, Beijing 100081, China
  • Jinlong Fan 3. Faculty of Geographical Sciences, Beijing Normal University, Beijing 100875, China
  • Feng Li 4. Shandong Provincial Climate Center, Jinan 250031, China
  • Hongying Li 5. Ningxia Institute of Meteorological Sciences, Yinchuan, Ningxia 750002, China
  • Jianguo Wei 6. Key Laboratory of Meteorological Disaster Monitoring, Early Warning, and Risk Management for Characteristic Agriculture in Arid Regions, China Meteorological Administration, Yinchuan, Ningxia 750002, China
  • Shujun Liu 1. College of Information Science and Engineering, Shandong Agricultural University, Tai’an 271018, Shandong, China
  • Huake Zhang 1. College of Information Science and Engineering, Shandong Agricultural University, Tai’an 271018, Shandong, China
  • Yanhui Jin 1. College of Information Science and Engineering, Shandong Agricultural University, Tai’an 271018, Shandong, China

Abstract

Accurate delineation of grape berry boundaries is essential for phenotypic measurement and growth assessment. This study proposes a multi-source feature fusion network (MFFNet) for instance segmentation of grape berries in dense clusters with frequent overlaps and blurred edges. MFFNet employs two parallel branches for feature extraction: a Swin Transformer backbone to capture hierarchical semantic features and an edge-detection branch that predicts an edge probability map to provide boundary cues. To address the substantial scale variation within a single image, the multilevel semantic features were enhanced using Adaptive Spatial Feature Fusion (ASFF). The edge probability map was introduced twice into the ASFF-enhanced multi-scale features. First, edge cues were injected into the highest-resolution fused feature map to strengthen global boundary awareness across the cluster. Second, during mask generation, edge cues were reintroduced within each candidate instance region to refine local contours and improve the separation in the adhered areas. Experiments on a custom dataset collected in Yinchuan, Ningxia, showed that MFFNet achieved an  of 93.4% and  of 93.4%, outperforming representative baselines, including Mask2Former and HTC. The proposed model remained stable on images with severe berry overlap and indistinct edges, supporting practical grape growth monitoring.      

Keywords: grape berry; instance segmentation; edge detection; multi-source feature fusion; precision agriculture

DOI: 10.25165/j.ijabe.20261902.10132

 

Citation: Yao M C, Yang X X, Zhang C M, Wu M X, Fan J L, Li F, et al. Multi-source feature fusion network for grape berry instance segmentation. Int J Agric & Biol Eng, 2026; 19(2): 294–302.

References

[1] Palacios F, Diago M P, Melo-Pinto P, Tardaguila J. Early yield prediction in different grapevine varieties using computer vision and machine learning. Precision Agriculture, 2023; 24（2）: 407–435.

[2] Lanza B, Botturi D, Gnutti A, Lancini M, Nuzzi C, Pasinetti S. A stride toward wine yield estimation from images: Metrological validation of grape berry number, radius, and volume estimation. Sensors, 2024; 24（22）: 7305.

[3] Liu S, Zeng X, Whitty M. A vision-based robust grape berry counting algorithm for fast calibration-free bunch weight estimation in the field. Computers and Electronics in Agriculture, 2020; 173: 105360.

[4] Coviello L, Cristoforetti M, Jurman G, Furlanello C. GBCNet: In-field grape berries counting for yield estimation by dilated CNNs. Applied Sciences, 2020; 10（14）: 4870.

[5] Buayai P, Yok-In K, Inoue D, Nishizaki H, Makino K, Mao X Y. Supporting table grape berry thinning with deep neural network and augmented reality technologies. Computers and Electronics in Agriculture, 2023; 213: 108194.

[6] Blasco J, Aleixos N, Cubero S, Gomez-Sanchis J, Molto E. Automatic sorting of satsuma （Citrus unshiu） segments using computer vision and morphological features. Computers and Electronics in Agriculture, 2009; 66（1）: 1–8.

[7] Bhargava A, Bansal A. Automatic detection and grading of multiple fruits by machine learning. Food Analytical Methods, 2020; 13（3）: 751–761.

[8] Bhargava A, Bansal A, Goyal V. Machine learning–based detection and sorting of multiple vegetables and fruits. Food Analytical Methods, 2022; 15（1）: 228–242.

[9] Gongal A, Amatya S, Karkee M, Zhang Q, Lewis K. Sensors and systems for fruit detection and localization: A review. Computers and Electronics in Agriculture, 2015; 116: 8–19.

[10] Ji W, Zhao D A, Cheng F Y, Xu B, Zhang Y, Wang J J. Automatic recognition vision system guided for apple harvesting robot. Computers and Electrical Engineering, 2012; 38（5）: 1186–1195.

[11] Zabawa L, Kicherer A, Klingbeil L, et al. Detection of single grapevine berries in images using fully convolutional neural networks. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops （CVPRW）, Long Beach: IEEE, 2019; pp.2571–2579. doi: 10.1109/CVPRW.2019.00313.

[12] Feng Q H, Xu X Z, Wang Z X. Deep learning-based small object detection: A survey. Mathematical Biosciences and Engineering, 2023; 20（4）: 6551–6590.

[13] Koirala A, Walsh K B, Wang Z, McCarthy C. Deep learning for real-time fruit detection and orchard fruit load estimation: Benchmarking of ‘MangoYOLO’. Precision Agriculture, 2019; 20（6）: 1107–1135.

[14] Shen Q, Zhang X, Shen M, et al. Multi-scale adaptive YOLO for instance segmentation of grape pedicels. Computers and Electronics in Agriculture, 2025; 229: 109712.

[15] Chen Y M, Li X, Jia M, Li J L, Hu T Y, Luo J. Instance segmentation and number counting of grape berry images based on deep learning. Applied Sciences, 2023; 13（11）: 6751.

[16] Zhang X, Gao Q M, Pan D, Cao P C, Huang D H. Research on spatial positioning system of fruits to be picked in field based on binocular vision and SSD model. In: 2020 International Conference on Computer, Information and Telecommunication Systems （CITS）, Jiaxing, China: IOP Publishing, 2021; 1748: 042011.

[17] Liang Q K, Zhu W, Long J Y, Wang Y N, Sun W, Wu W N. A real-time detection framework for on-tree mango based on SSD network. In: 11th International Conference on Intelligent Robotics and Applications （ICIRA 2018）, Singapore: Springer, 2018; pp.423–436. doi: 10.1007/978-3-319-97589-4_36.

[18] Sa I, Ge Z Y, Dayoub F, Upcroft B, Perez T, McCool C. Deepfruits: A fruit detection system using deep neural networks. Sensors, 2016; 16（8）: 1222.

[19] Wan S, Goudos S. Faster R-CNN for multi-class fruit detection using a robotic vision system. Computer Networks, 2020; 168: 107036.

[20] Tu S Q, Pang J, Liu H F, Zhuang N, Chen Y, Zheng C, et al. Passion fruit detection and counting based on multiple scale faster R-CNN using RGB-D images. Precision Agriculture, 2020; 21: 1072–1091.

[21] Mai X C, Zhang H, Meng M Q H. Faster R-CNN with classifier fusion for small fruit detection. In: 2018 IEEE International Conference on Robotics and Automation （ICRA）, Brisbane, Australia: IEEE, 2018; pp.7166–7172. doi: 10.1109/ICRA.2018.8461130.

[22] Santos T T, de Souza L L, dos Santos A A, Avila S. Grape detection, segmentation, and tracking using deep neural networks and three-dimensional association. Computers and Electronics in Agriculture, 2020; 170: 105247.

[23] He K M, Gkioxari G, Dollár P, Girshick R. Mask R-CNN. In: 2017 IEEE International Conference on Computer Vision （ICCV）, Venice, Italy: IEEE, 2017; pp.2980–2988. doi: 10.1109/ICCV.2017.322.

[24] Tian Y N, Yang G D, Wang Z, Li E, Liang Z Z. Instance segmentation of apple flowers using the improved mask R-CNN model. Biosystems Engineering, 2020; 193: 264–278.

[25] Shen L, Su J Y, Huang R, Quan W M, Song Y Y, Fang Y L, et al. Fusing attention mechanism with Mask R-CNN for instance segmentation of grape cluster in the field. Frontiers in Plant Science, 2022; 13: 934450.

[26] Jia W K, Tian Y Y, Luo R, Zhang Z H, Lian J, Zheng Y J, et al. Detection and segmentation of overlapped fruits based on optimized mask R-CNN application in apple harvesting robot. Computers and Electronics in Agriculture, 2020; 172: 105380.

[27] Huang Y P, Wang T H, Basanta H. Using fuzzy mask R-CNN model to automatically identify tomato ripeness. IEEE Access, 2020; 8: 207672–207682.

[28] Liu Y, Sun P, Wergeles N, Shang Y. A survey and performance evaluation of deep learning methods for small object detection. Expert Systems with Applications, 2021; 172: 114602.

[29] Huang X, Peng D D, Qi H N, Zhou L, Zhang C. Detection and instance segmentation of grape clusters in orchard environments using an improved mask R-CNN model. Agriculture, 2024; 14（6）: 918.

[30] Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv, arXiv: 2010.11929. doi: 10.48550/arXiv.2010.11929.

[31] Liu Z, Lin Y T, Cao Y, Hu H, Wei Y X, Zhang Z. Swin transformer: Hierarchical vision transformer using shifted windows. In: 2021 IEEE/CVF International Conference on Computer Vision （ICCV）, Montreal, QC, Canada: IEEE, 2021; pp.9992–10002. doi: 10.1109/ICCV48922.2021.00986.

[32] Liu H, Wang X X, Zhao F Y, Yu F Y, Lin P, Gan Y, et al. Upgrading swin-B transformer-based model for accurately identifying ripe strawberries by coupling task-aligned one-stage object detection mechanism. Computers and Electronics in Agriculture, 2024; 218: 108674.

[33] Xiao B J, Nguyen M, Yan W Q. Fruit ripeness identification using transformers. Applied Intelligence, 2023; 53（19）: 22488–22499.

[34] Song K, Yang J, Wang G. A Swin transformer and MLP based method for identifying cherry ripeness and decay. Frontiers in Physics, 2023; 11: 1278898.

[35] Zheng H, Wang G H, Li X C. Swin-MLP: A strawberry appearance quality identification method by Swin Transformer and multi-layer perceptron. Journal of Food Measurement and Characterization, 2022; 16（4）: 2789–2800.

[36] Wang J H, Zhang Z Y, Luo L F, Wei H L, Wang W, Chen M Y, et al. DualSeg: Fusing transformer and CNN structure for image segmentation in complex vineyard environment. Computers and Electronics in Agriculture, 2023; 206: 107682.

[37] Lu S L, Liu X Y, He Z X, Zhang X, Li W B, Karkee M. Swin-Transformer-YOLOv5 for real-time wine grape bunch detection. Remote Sensing, 2022; 14（22）: 5853.

[38] Liu G X, Zhang Y H, Liu J, Liu D Y, Chen C L, Li Y J, et al. An improved YOLOv7 model based on Swin Transformer and Trident Pyramid Networks for accurate tomato detection. Frontiers in Plant Science, 2024; 15: 1452821.

[39] Charisis C, Argyropoulos D. Deep learning-based instance segmentation architectures in agriculture: A review of the scopes and challenges. Smart Agricultural Technology, 2024; 8: 100448.

[40] Zhang Z J, Fu H Z, Dai H, Shen J B, Pang Y W, Shao L. ET-Net: A generic edge-attention guidance network for medical image segmentation. In: 22nd International Conference on Medical Image Computing and Computer Assisted Intervention （MICCAI 2019）, Shenzhen, China: Springer, 2019; pp.442–450. doi: 10.1007/978-3-030-32239-7_49.

[41] Zhao J X, Liu J J, Fan D P, Cao Y, Yang J F, Cheng M M. EGNet: Edge guidance network for salient object detection. In: 2019 IEEE/CVF International Conference on Computer Vision （ICCV）, Seoul, Korea （South）: IEEE, 2019; pp.8778–8787. doi: 10.1109/ICCV.2019.00887.

[42] Yang J Y, Jiao L C, Shang R H, Liu X, Li R Y, Xu L C. EPT-Net: Edge perception transformer for 3D medical image segmentation. IEEE Transactions on Medical Imaging, 2023; 42（11）: 3229–3243.

[43] He C, Li S L, Xiong D H, Fang P Z, Liao M S. Remote sensing image semantic segmentation based on edge information guidance. Remote Sensing, 2020; 12（9）: 1501.

[44] Sheng X, Kang C M, Zheng J Y, Lyu C. An edge-guided method to fruit segmentation in complex environments. Computers and Electronics in Agriculture, 2023; 208: 107788.

[45] Jia W, Cao K, Liu M Y, Lu Y Q, Ji Z, Liu G L, et al. FCOS-EAM: An accurate segmentation method for overlapping green fruits. Computers and Electronics in Agriculture, 2024; 226: 109392.

[46] Zimmermann R S, Siems J N. Faster training of Mask R-CNN by focusing on instance boundaries. Computer Vision and Image Understanding, 2019; 188: 102795.

[47] Yang C C, Geng T Y, Peng J, Xu C, Song Z C. Mask-GK: An efficient method based on mask Gaussian kernel for segmentation and counting of grape berries in field. Computers and Electronics in Agriculture, 2025; 234: 110286.

[48] Liu S T, Huang D, Wang Y H. Learning spatial fusion for single-shot object detection. arXiv, 2019; arXiv: 1911.09516. doi: 10.48550/arXiv.1911.09516.

[49] Du W S, Liu P. Instance segmentation and berry counting of table grape before thinning based on AS-SwinT. Plant Phenomics, 2023; 5: 0085.

[50] Li H, Xiong X R, Liu C X, Ma Y, Zeng S, Li Y Q. SFFNet: Staged feature fusion network of connecting convolutional neural networks and graph convolutional neural networks for hyperspectral image classification. Applied Sciences, 2024; 14（6）: 2327.

[51] Li S, Bitsch L, Hanf J. Grape supply chain: Vertical coordination in Ningxia, China. Asian Journal of Agriculture, 2021; 5（1）: 050103.

[52] Chen R W, Zhang X Y, Yang Y, Yang Y E, Wang J, Li H Y. Analyses of vineyard microclimate in the eastern foothills of the Helan Mountains in Ningxia region, China. Sustainability, 2023; 15（17）: 12740.

Downloads

Published

2026-05-21

How to Cite

(1)
Yao, M.; Yang, X.; Zhang, C.; Wu, M.; Fan, J.; Li, F.; Li, H.; Wei, J.; Liu, S.; Zhang, H. Multi-Source Feature Fusion Network for Grape Berry Instance Segmentation. Int J Agric & Biol Eng 2026, 19, 294-302.

Issue

Vol. 19 No. 2 (2026): IJABE

Section

Information Technology, Sensors and Control Systems

License

Copyright (c) 2026 International Journal of Agricultural and Biological Engineering

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

IJABE is an international peer reviewed, open access journal, adopting Creative Commons Copyright Notices as follows.

Authors who publish with this journal agree to the following terms:

  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).