Estimation of crop yield along its different growth stages is essential when making decisions about disease management, harvest, storage, transport, and marketing. In the case of cacao (Theobroma cacao) yield estimation is typically based on manual counts, which are time-consuming, expensive, and often associated to huge estimation errors. Indeed, because of the unsynchronized pattern of flowering and yield development phases in cocoa trees, the estimation of yield and production rely on large sampling of individual trees on a weekly basis. Designing a low-cost machine vision system with strong operability for the real-time identification of cocoa pods in different growth stages under natural environments is of great significance for research and cocoa industry. Accurate cocoa pods detection is essential for several applications such as fruit counting, yield estimation and precision agriculture. The main goal of this study was to propose a simple and reliable method for cocoa pods segmentation and counting from RGB images acquired with mobile phones, using deep learning techniques. The method presented here involves four steps, namely (i) image acquisition, (ii) image annotation, (iii) training models [preprocessing and applied training technique] and (iv) model validation. For step 1, 794 images from cocoa fields located in Ivory Coast and in Cameroon were obtained with the main camera of a mobile device. For step 2, cocoa pods were annotated from images using the Zooinverse collaborative platform. For step 3, three deep learning recognition models, Faster-RCNN, Mask-RCNN and YOLOv3 were trained using 80% of the RGB images and annotations. The applied variants of Faster-RCNN were, RCNN-R-50-C4-3x, RCNN-R-101-C4-3x, RCNN-R-50-FPN-3x and RCNN-R-101-FPN-3x while R101-C4-3x and R101-FPN-3x were used for Mask-RCNN. For step 4, 10% of data were used for testing and 10% for validating the trained deep learning methods. Results showed that, among the different models compa