Background

Liquid handling is an important part of many experiments in fields related to biology and chemistry, and is frequently used in genomic or proteomic research. In general, liquid handling used in such experiments requires accuracy and precision, but is also a very tedious task requiring a considerable amount of time if carried out manually [1,2,3,4,5,6,7,8]. Therefore, researches on automation of liquid handling have become paramount [1, 3, 5,6,7]. Most automated liquid handler (ALH) systems use a conveyor belt to optimize fast inspection of multiple samples at a large hospital. However, such systems require a complicated set of equipment for diagnosis and analysis, and hence can only be properly installed in large central laboratories that has access to various devices. Therefore it is difficult to implement the ALH system in develo** countries where lack of central laboratories. Even in developed countries, small hospitals find it difficult to apply this system, resulting in manual clinical tests. As an alternative, small hospitals may send samples to a central laboratory equipped with the appropriate system, but this will result in patients feeling uncomfortable due to the slow turnaround time of the test results. To address these problems, a portable clinical test system using robotic automation was recently developed [4]. Despite the throughput might be smaller than that of a conventional system, it is relatively flexible, small, and inexpensive. Moreover, such a point-of-care test (POCT) device performs the task of a portable clinical test while being suitable to carry out the selective diagnosis work required at small hospitals [9,21].

The rest of this paper is organized as follows: We describe the materials and methods in “Methods” section, the results and the discussion are presented in “Results” and “Discussions” sections, respectively, and the conclusion is followed in “Conclusions” section.

Methods

A commercial automated system (IChroma™ Smart Reader, Boditech Med Inc.) shown in Fig. 1 was modified for the experiments in this paper. Before starting the analysis, the cartridge is inserted in the middle right part of the equipment shown in the image to the left of the figure. The cartridge contains the samples to be analyzed and the various reagents required. The left image of the figure is the enlarged view of the cartridge mounting part. As you can see in this figure, a pipette tip is put together with the cartridge. This device measures the concentration of the target substance in blood or blood serum of specimens including human, by using a specially designed cartridge. For the diagnosis, a pipette tip should be inserted manually in addition to the cartridge. The tip and the tip holder are the most important components in the device, and any malfunction of those components will lead to an inaccurate diagnosis result. After mounting the reagent and tip in the cartridge, the user inserts the cartridge in the device and presses the start button to initiate the diagnostic analysis. The tip is first mounted onto the tip holder of the pump installed within the device, and the reagents are pumped in/out to/from various chambers in the cartridge. For an accurate diagnosis, malfunctions such as the absence of the tip, misalignment of the tip and holder, and inaccurate amount of the reagent loaded into the tip must be identified and resolved during the diagnosis process. To perform these checks, the smartphone camera (PO1150K, Pixelplus, Co., Ltd.) was mounted in such a way that the tip holder of the pump would be positioned at the middle of the horizontal axis in the image. The employed camera shown in the left of Fig. 2 is an inexpensive module commonly used in smartphones. The right image of the figure shows the example captured using the camera.

Fig. 1
figure 1

IChroma™ Smart Reader (left) and the cartridge loading example (right)

Full size image
Fig. 2
figure 2

PO1150K Smart Camera module (left) and the image of the tube loaded with reagent (right)

Full size image

The image processing functions are implemented in the host Android platform, where the LCD and touch panel are located in front of the device as shown in the left image of Fig. 1. Simple algorithms are preferred considering the computational power of the host. In our previous work, the binarization with a fixed threshold and the projections of the binary image were sufficient to successfully detect the absence of the tip or the holder. In this work, we focus on the monitoring of the pump operation.

First, the relation between the pump steps and the reagent mass loaded to the tip was analyzed in order to verify the accuracy and precision of the pump. The reagent mass loaded were measured every 40 steps from 0 to 640 steps with a chemical balance (ARG222, Ohaus Corp., USA). The loading and measurement for precision testing were repeated 5 times for each step. The mass of the tip and tip cover were measured before loading and subtracted from the total mass of the loaded tip and tip cover. The tip and the cover were changed for every measurements because it is difficult to remove the reagent totally from the tip. The loaded tip was imaged before being covered and unmounted from the holder to measure the mass.

Figure 3 shows the whole flow of the image processing for pipette monitoring. Given that the horizontal axis position of the tip holder is constant, the region of interest (ROI) was set centered to that position for the image acquired. Figure 4a shows an example of ROI images. Since red-colored water was used to emulate the blood samples in this study, the reagent area was extracted from the pure red image (Y-R image block in Fig. 3). The pure red image was obtained by subtracting the red channel to the luminance image (Fig. 4b, c). It was binarized with a fixed threshold after 3 × 3 median filtering. Binarization with a fixed threshold was sufficient as in our previous work, because the imaging condition such as illumination and background matte finish could be sufficiently controlled over production.

Fig. 3
figure 3

Schematic of flow algorithm for image processing

Full size image
Fig. 4
figure 4

a ROI of entire tip image; b the luminance image; c the pure red image; d the extracted reagent area

Full size image

The extracted reagent area shown in the left of Fig. 5 was projected horizontally and the resultant projection image in the right of the figure was analysed. The support length of the projection image was calculated by scanning the projection vertically and counting the non-zero valued vertical positions. As the support length was related to the reagent volume, the support length was referred to the volume length as shown in the right image of Fig. 5. Note that the actual reagent volume was proportional to the cube of the volume length because of the conical shape of the tip. Various sophisticated methods for binary image filtering and calculation of the projection support can be applicable if the above simple procedure cannot achieve correct operation. However, the proposed simple procedure alone was sufficient for our experiments.

Fig. 5
figure 5

The horizontal projection of the reagent area (left) and the volume length defined on the projection image (right)

Full size image

Results

The average and standard deviation of the reagent mass for each step is shown in Table 1, where the coefficients of variation (CV) values are shown in the last row. The CV values were less than 5.7 ppm, indicating that the pump loaded the precise amount of reagents. By regression analysis, the CV values were not related to the steps showing that the precision was independent of the pump steps. The coefficient of determinant (R2) which represents the linearity between the average masses and pump steps was ‘1’ as shown in Fig. 6, verifying the accuracy of the pump.

Table 1 The linearity between the number of motor steps and reagent mass (five experiments per each number of steps)
Full size table
Fig. 6
figure 6

The linearity between the number of pump steps and average reagent mass (fitted linear equation: y = 18.4x + 0.001, R2 = 1.000)

Full size image

Table 2 shows the statistics for the relation between the pump steps and volume lengths calculated from the image processing. In accordance to Table 2, the pump showed high precision having CV values of less than 3.1% and was independent from the pump steps. Figure 7 depicts the relation between the pump steps and volume lengths. As the shape of the tip was a cone, the volume of the reagent was proportional to the cube of the volume length as shown in the figure. Therefore the relation between the pump steps and the cubes of the volume lengths were highly linear as shown in Fig. 8 (R2 = 0.996).

Table 2 The relation between the number of motor steps and volume length from image processing
Full size table
Fig. 7
figure 7

The relation between the number of pump steps and average volume length

Full size image
Fig. 8
figure 8

The linearity between the pump steps and cube of the average volume length (fitted linear equation: y = 30337x + 500.8, R2 = 0.996)

Full size image

The pump steps could be estimated by the linear relation shown in Fig. 8. The statistics of the estimation error is summarized in Table 3. Note that the standard deviation of the estimation of pump steps shown in the 2nd row was not from the average of the error but from the true steps shown in the 1st row. This reasoning was from the assumption that the average error might converge to zero with more experiments. As the pump step becomes larger, the standard deviation becomes larger, so it is more reasonable to investigate the pump performance by examining the relative standard deviation. The relative standard deviation is usually divided by the average of the standard deviation, but is divided into a true step instead of an average with the same reasoning as described above. The 3rd row of the table showed the relative standard deviations, and they were less than 9.4% except that for the smallest pump step. Furthermore it was less than 4.5% for the steps larger than 240 steps, which are closer to the actual sample volumes. These results demonstrate that the pump malfunction can be monitored by the proposed scheme using the camera for smart devices. Possible and simple decision of the pump malfunction can be done when the difference between the pump steps and the estimates from the volume lengths break the predefined bounds.

Table 3 The pump step estimation error statistics
Full size table

In our previous work, the ROI image of the tip and tip holder was binarized through a similar procedure. The binary images for the tip and the holder were separated and projected vertically and horizontally, respectively. The center position and the top of the holder were calculated from the supports of the projections as in this work. The calculated positions of the tip or the holder were very linear to the pertinent stepper motor’s positions (the coefficients of determination are 0.997 and 0.999 for the tip and the holder, respectively). The cubes of the volume lengths from the presented method in this work were also highly linear to the actual reagent volume, showing the consistency of the results with the previous work.

Discussions

In this study, an image-based monitoring method for pum** operation was introduced for diagnostic devices with automated pipetting. The support length determined from the horizontal projection of the reagent area was a good estimate of the pump steps or reagent volume. The simple image processing which includes the reagent area separation by the binarization of a specific color image with a fixed threshold, the vertical project of the binary image, and the support length calculation of scanning the projection for non-zero pixel, was sufficient to deliver the estimate. The proposed method can be applied to the detection of malfunctions of any automatic pipette device when implemented in conjunction with previous studies of misalignment detection of the tip member and tip holder.

The proposed simple image processing method using a commercial camera sensor exhibited sufficient performance at a low cost for solving the verification problems that can occur in other POCT devices as well. Owing to the widespread use of many high-performance camera sensors, this method can be applied to other similar devices.

By applying the proposed simple image processing methods to a device with a highly controllable imaging condition, user mistakes and/or device malfunction can be prevented by detecting device failure. Especially the volume estimation method presented in this study will serve for the maintenance of the pump which is one of the most important component for the devices with automated pipetting function.

The suggested image processing using the volume verification to monitor pum** operations can be employed to devices with the inaccuracy specification of less than 5%, if the dispensing target is elongated in the vertical direction as the tip. Even if the dispensing target of the device is flat such as the microtiter, this method can still be applied if there is a long reservoir between the dispenser and the target where the volume can be measured. In case that the device has an accurate and precise dispenser (CV < 5.7 ppm) as in this work, the detection of the pum** operation malfunction will be more effective rather than trying to verify the volume. Comparison studies on the malfunction detection performance between several methods was not given in this work, since it should be measured for numerous devices in production. Note that the performance of this kind of detection method is highly related to the production yield. Therefore the applicability of the presented method should be demonstrated just prior to commercialization.

Conclusions

In this study, an image-based failure monitoring method was introduced for POCT devices with automated pipetting. The simple image processing method using a commercial camera sensor can be detect the malfunctions of POCT devices such as the tip absence or misalignment of the tip and the tip holder. Especially the presented image processing also monitors pum** operations with the inaccuracy specification of 5%.