Background

Whiteflies are insects classified in the Aleyrodidae family and consist of more than 1500 species [21]. Their presence is sufficient to cause serious crop yield loss, e.g., damage by Bemisia tabaci (Gennadius, 1889), a very invasive whitefly species. It takes approximately 20 days during warm weather conditions for a whitefly to develop from an egg to a crawler, through to pupae, and finally an adult. Female whiteflies originate from fertilized eggs, while males originate from unfertilized eggs; the typical sex ratio is 2:1, females to males [33]. Whiteflies are phloem feeders, which means they use their four interlocked stylets to enclose food and a salivary canal allowing independent movements between plant mesophyll cells [27]. While feeding, they secrete saliva as a lubricant during the penetration of the stylets, which contain enzymes and metabolites, thereby providing protection against plant wound response [13]. Furthermore, whiteflies indirectly cause damage to plants by acting as a vector of viruses, including more than 100 plant viruses, such as those in the Begomovirus genus, which causes tomato yellow leaf curl virus, and viruses in the Crinivirus genus, which causes tomato chlorosis virus [17]. Whiteflies are predominantly polyphagous, e.g., B. tabaci has a wide host range such as tomato, cucumber, cotton, and sweet potato [30]. The two most dominant B. tabaci biotypes are Middle East-Minor Asia 1 (MEAM1 or biotype B) and Mediterranean (MED or biotype Q) genetic groups. These two biotypes have caused serious yield losses in more than 60 countries [10, 20, 29, 34, 36]. The most common methods employed to test whitefly host compatibility in greenhouse and laboratory settings are choice and no-choice assays [35]. Choice assays allow whiteflies to choose between several plant genotypes to assess whitefly preference, while no-choice assays limit whiteflies to feed on a single plant to evaluate response. A choice assay can be prepared using a Y-tube olfactometer setup for whole plants or leaves, which evaluates relative preference based on volatile compounds as well as other phenotypic characteristics such as insect mortality and oviposition [16]. On the other hand, a no-choice assay can be performed by using clip-on cages on leaves [3].

Two of the most important observation parameters obtained from both choice and no-choice assays are adult insect survival and oviposition rate. Oftentimes, the quantification of adult survival and oviposition are manually done by looking through a stereomicroscope [38]. Counting adult survival in a clip-cage setting is quick but counting the number of eggs deposited by the insects on each leaf is very laborious. Each whitefly egg is approximately 0.2 mm in length and 0.08 mm in width while its color ranges from translucent green to brown, depending on maturity. On a susceptible potato line, 5 female whiteflies, inside a 2 cm diameter clip-cage, can lay more than 100 eggs in 5 days [25]. In a natural setting, whiteflies deposit eggs at the abaxial of the leaves, thereby hiding each egg from predators and environmental factors such as precipitation and high-intensity ultraviolet light [26]. Moreover, some leaves have thick veins, trichomes, and surface unevenness that may conceal eggs and hinder observation. Furthermore, whitefly eggs are commonly found in leaf regions surrounded by whitefly honeydew, which hinders observation [37]. Whitefly honeydew provides optimal conditions for mold to grow and as protection for the development of whitefly eggs to instars [8]. Due to the aforementioned factors, researchers would benefit from a more reliable and systematic protocol for analyzing whitefly assay samples.

To assure the reliability of performed assays, microscopic images are stored for further analysis. One of the ways to analyze microscopic images is by digital image processing. Digital image processing is a computerized method for automatically identifying and detecting characteristics of objects in an image by performing operations such as color conversion, edge detection, color segmentation, and blob analysis. Digital image processing has been employed in microscopic image analysis such as for mosquito egg counting [11, 19], and beetle egg counting [12]. In the works mentioned, it involves a user that specifies algorithm parameters to optimize the image analysis results; therefore, considering it as a semi-automated approach. Unlike digital image processing-based algorithms, a deep learning-based algorithm is more resistant to variations in image appearance and requires less input from a user. Deep learning is a subset of artificial intelligence that aims to train a neural network model to learn feature hierarchies from a given dataset. Deep learning models are capable of accurately detecting minute objects, such as insect eggs, even without manually tuning algorithm parameters. Deep learning has been used in microscopic image analysis for counting nematodes [1, 18], stomata [9], and protozoan parasites [39].

This work aims to develop a novel and more efficient protocol for automated whitefly egg quantification to accelerate the determination of insect-resistant and susceptible plant accessions. The specific objectives are: (1) the development of a fast and accurate whitefly egg quantification algorithm; (2) designing a web-based platform to deploy the algorithm; (3) assessing the advantages and disadvantages of using different imaging setups for collecting leaf image assays; and (4) determining plant–insect resistance and susceptibility based on the automatically obtained egg counts. This work proves the benefits of employing novel computer techniques to achieve more objective research results in the field of plant–insect resistance.

Materials and method

Plant materials and growth conditions

Wild (Solanum berthaultii (Hawkes, 1963)—BER481-3) and cultivated (S. tuberosum cv. RH89-039-16) potatoes were obtained from the Wageningen University and Research (WUR) Plant Breeding collection, Wageningen, The Netherlands. For collecting leaf images, in vitro propagated cuttings were grown for 2 weeks in MS20 medium, and successively transferred to the greenhouse in ⌀14 cm pots with potted soil for 3 weeks before the whitefly infection assays. Resistant wild tomato plants (S. habrochaites (Knapp & Spooner, 1999)—LA1777) and susceptible cultivated tomato plants (S. lycopersicum cv. Moneymaker) were also used in this study. Seeds were obtained from WUR Plant Breeding Department, Wageningen, The Netherlands. Seeds were sown on germination media for 2 weeks before being transplanted into ⌀14 cm pots with potting soil for 3 weeks before the whitefly infestation assays were commenced. The plants were grown in peat soil in an insect-proof greenhouse at Unifarm with a 16 h light and 8 h dark photoperiod, 21 °C/19 °C (day/night) and 70% relative humidity from September–October 2022 in Wageningen, The Netherlands.

Whitefly assay

Whitefly assays were conducted on 5-week-old plants; 3 plants per genotype. Non-viruliferous whiteflies (B. tabaci group Mediterranean-Middle East-Asia Minor I), reared on S. lycopersicum cv. Forticia from the WUR Plant Breeding Department were used for screening. No-choice assays were carried out in an insect greenhouse of WUR. The assay was done by attaching two clip-on-cages ⌀2 cm containing five synchronized 1-day-old female B. tabaci whiteflies on the abaxial side of the second and third fully expanded leaf of each plant. Five days later, the leaves attached with clip-cages were harvested for image acquisition, egg quantification (OR) and adult survival (AS) on the same day. Leaves can optionally be stored on wet filter paper and in 4 °C to maintain cell turgidity and prevent dehydration. AS and OR for tomato plants [20] were calculated according to the following equations, respectively:

$$ {\text{AS}} = \left( {\frac{alive\;whiteflies}{{total\;whiteflies}}} \right)\;survival \;for\;5\;days, $$
(1)
$$ {\text{OR}} = \frac{2 \times number\;of\;eggs}{{alive\;whiteflies + total\;whiteflies}}eggs\;female^{ - 1} \;for\;5\;days. $$
(2)

Arcsine transformation was used to normalize AS, whereas square root transformation was used for oviposition rate. Statistical analyses via t-tests were performed using Python 3.8.13, with the support of SciPy scientific computing library v1.9.3.

Leaf image acquisition

The leaf image samples used in the whitefly assay were acquired using two different imaging setups: (1) using a commercial microscope (VHX-7000) (Keyence, Japan), and 2) using AutoEnto device, as shown in Fig. 1. The VHX-7000 is a 4K digital microscope designed for surface microscopy. It acquires high resolution leaf images by acquiring tiled images based on box vertices that were manually selected using its accompanying controller. The tiled images were automatically stitched together by its software. The VHX-7000 was electronically adjusted to set a 45–50 mm distance from the lens to the camera while the magnification was set to 30×. The automatic white balance and brightness were manually adjusted and kept uniform throughout the trials.

Fig. 1
figure 1

Workflow for rapid determination of plant resistance based on insect oviposition

Full size image

Meanwhile, the AutoEnto device is an imaging system developed by the Greenhouse Horticulture & Flower bulbs Business Unit of Wageningen Plant Research in Bleiswijk, The Netherlands. It is designed to rapidly acquire images of tiny biological samples such as insects, insect eggs, nematodes, and alike. It is equipped with an industrial CMOS color camera (IDS Imaging Development Systems GmbH, Germany), replaceable C-mount lenses, and a custom x–y table that can hold up to 9 dishes, for automated image acquisition and analysis. In this work, it was configured with a 30× magnification C-mount lens (Kowa Company, Japan), acquiring 4912 × 3684 pixels per image with a spatial resolution of 350 pixel/mm. In this research, it was configured to take 35 images with 4912 × 3684 pixels over a 5 × 7 grid that were stitched together into a single image. The AutoEnto device was only used for image acquisition but not for image analysis.

Automatic egg quantification algorithm

The acquired leaf images were analyzed using an automatic egg quantification algorithm, as illustrated in Fig. 2. The egg quantification algorithm was developed using Python 3.8.3 programming language, with the support of OpenCV image processing library [5], PyTorch deep learning library [24] and MLFlow machine learning tracking library [

Fig. 2
figure 2

Automated egg quantification algorithm

Full size image

Dataset preparation and image pre-processing

A leaf image dataset was prepared using the image pre-processing step of the algorithm. First, the size of the leaf image is reduced from L × W to L − SL × W − SW based on a pre-defined tiling size, where L and W are the length and width of the leaf image while SL and SW are the surplus length pixels and surplus width pixels, respectively; this was done by equally removing the pixels from all sides of the leaf image. Removal of the surplus pixels was done to attain an equal tiling size. In this work, a tiling size of 1400 × 1400 pixels was used on images obtained from both setups, with a padding of 100 pixels on all sides of the tiled leaf image. Tiling is a technique in deep learning which cuts an image into several equal parts to achieve better object detection results [28]. On the other hand, the padding allows the merging of duplicate egg detections after obtaining the detection results from each tiled leaf image later. The tiling process produces n 1600 × 1600 tiled leaf images that are used as individual inputs to the deep learning-based object detection model. Every egg on each tiled leaf image was annotated using a rectangular box via Darwin v7 image annotation platform with the assistance of experts. The statistical summary of the prepared image dataset is shown in Table 1.

Table 1 Image dataset statistical information
Full size table

Object detection and image stitching

The automated egg quantification algorithm uses a YOLOv5m (Ultralytics, Los Angeles, CA, USA) deep learning model for detecting the eggs from each tiled leaf image. YOLOv5m is an object detection model with three components: backbone, neck, and head. It uses cross-stage partial networks as the backbone for feature extraction while the neck, made of path aggregation networks, combines the extracted features. The combined features are used as input to a YOLO layer, which acts as the model’s head, for obtaining bounding box predictions. In this work, the YOLOv5m was specifically chosen since it has a good balance of speed and performance compared to the other YOLOv5 variants. The YOLOv5m model’s input was set to the default 614 × 614 pixels, thereby resizing each 1600 × 1600 tiled image to 614 × 614 pixels for inference. The YOLOv5m model’s output was defined to have a single image class, egg class. After detecting the eggs from each tiled leaf image, the leaf image was stitched back together while retranslating the bounding box coordinates according to the original leaf image size.

Detection post-processing

Three detection post-processing methods were applied to reduce the algorithm errors: detection merging, object size filtering, and confidence thresholding. In detection merging, similar objects found in adjacent tiled images were merged using Intersection-over-Union (IoU). IoU is a measure of the overlap between two objects in an image. IoU values closer to 1 indicate higher overlap and 0 otherwise. IoU was computed using Eq. 3:

$$ IoU = \frac{{area\left( {B_{1} \cap B_{2} \cap \ldots B_{o} } \right)}}{{area\left( {B_{1} \cup B_{2} \cup \ldots B_{o} } \right)}}, $$
(3)

where Bo is the bounding box coordinates of each detected object, with o as the object index. Bo includes four coordinates: x1, y1, x2, and y2, where x1 and y1 belong to the object’s x and y vertex box coordinates, and x2 and y2 belong to the vertex opposite to x1 and y1. If the IoU of two or more egg detection boxes is greater than 0.5, the boxes are merged by retaining the lowest x1 and y1 and highest x2 and y2 and counting the overlap** objects as a single object.

The object size filtering threshold was manually determined using the average size of each detected object based on Table 1. If the length or width of a detected object is less than 20 pixels, then the detected object was ignored. Such detected objects are trichomes and leaf spots that resemble the appearance of whitefly eggs. If the length or width of a detected object is more than 90 pixels, then the detected object was also ignored since such objects may include nymphs or other unwanted objects.

Confidence thresholding utilizes the confidence score of each detection, which ranges from 0 to 1.0, where values closer to 1.0 indicate higher confidence. To ignore detected objects with low confidence scores, a pre-set classification confidence threshold was defined and fine-tuned. Some of the detected objects with low confidence scores include trichomes and leaf spots.

Algorithm evaluation

The algorithm was evaluated using two methods: object-level testing and image level testing, with the test dataset defined in Table 1. In this context, object-level testing refers to how the model performs when tested on each object of all the leaf images. On the other hand, image-level testing refers to the overall algorithm performance when tested on individual leaf images. In object-level testing, the algorithm performance was evaluated by automatically matching each set of ground truth box coordinates with each predicted box coordinates via IoU. If the calculated IoU between two paired coordinates was higher than 0.5, it was counted as a true positive (TP) detection. All unmatched ground truth box coordinates were considered as missed detections. Using the above definitions, the following metrics were calculated:

$$ Precision = \frac{TP }{{total\;number\;of\;detected\;egg\;objects}}, $$
(4)
$$ Recall = \frac{TP }{{true\;number\;of\;egg\;objects}}, $$
(5)
$$ F_{1} score = 2 \cdot \frac{precision \cdot recall}{{precision + recall}}. $$
(6)

F1-score is a performance metric that balances precision and recall; it ranges from 0 to 1, where values closer to 1 indicate better performance. Meanwhile, the miss rate measures how many detections were undetected by the algorithm; it ranges from 0 to 1, where values closer to 1 indicate worse performance.

In image-level testing, counting accuracy, miss rate, and coefficient of determination (r2) were measured. Counting accuracy was measured by obtaining the percent difference between the true counts (TC) and the automatic counts (AC) per leaf image, as follows:

$$ Counting\;accuracy = \frac{TC - AC}{{TC}}. $$
(7)

Meanwhile, the miss rate is the ratio of missed detections and the true number of egg objects in a leaf image, computed as follows:

$$ Miss\;rate = \frac{missed\;detections\;in\;a\;leaf\;image}{{true\;number\;of\;egg\;objects\;in\;a\;leaf\;image}}. $$
(8)

Finally, r2 measures the performance of the model relative to the manual counts.