On Mass Estimation of fruits with modern Computer Vision: a Deep Learning approach

This work presents a comparison between a selection of Deep Learning models against a selection of Machine Learning models studying how these perform estimating the mass of a selection of fruits using Computer Vision techniques to extract features for this purpose. The selected models for Deep Learning include a Fully Connected Neural Network, a Fully Connected Neural Network with a dropout factor between hidden layers, a proposed model with a similar architecture, a simple CNN model, MobileNet, and DenseNet; while the Machine Learning selection includes SVR and Linear Regression. All these were implemented using TensorFlow and Scikit-Learn libraries in Python 3.9. YOLOv8, Detectron2, and the manually annotated segmentations were used to separate the fruits from the background of the image to extract features and compare their results. This work also presents the creation of three datasets of the selection of fruits, in this case, blueberries, raspberries, and strawberries. Two datasets were created for each fruit, one relating high-definition images of the fruits with high-precision readings of their mass, and another one including only images of the fruits. The general aim of this work is to find out, how to build datasets of fruits with high-definition images and high-precision readings of their mass, which method is better to extract features of the fruits of the images by segmenting the image between YOLOv8 and Detec- tron2, and how the selection of Deep Learning models performs against the selection of Machine Learning models estimating the mass of the selection of fruits. Currently, the best results estimating the mass of fruits and similar structures have been obtained by analyzing simple features with Machine Learning models, which is why a comparison with Deep Learning is of special interest. After the creation of the datasets, both YOLOv8 and Detectron2 models were compared showing high accuracy in segmenting the fruits from the images. It was found that the best correlation feature with mass is the area of the fruit, which is proportional to the real area. After extracting features, these were divided into classes which were contrasted in the results of all models against all classes and with different training epochs when possible; resulting in 288 different experiments. The experiments resulted in Machine Learning outperforming Deep Learning, however, Deep Learning presents the potential improvement under different conditions such as more training epochs or larger datasets. The results obtained from the creation of the datasets, the segmentation of the fruits, and the comparison of models estimating the mass of fruits, are of interest to research lines aiming to create datasets relating images and physical properties of objects, perform segmentation of non-regular polygon structures in 2D images, and estimate the mass of fruits or similar structures.

0000-0003-0966-5685