Keywords

1 Introduction

Feed is one of the most important variable cost in aquaculture, and it can even reach 80% in some cultured species [1,2,3]. So it is important to minimum the feed cost [4]. It not only reduces the feed waste and save costs, but also reduces the environmental pollution caused by residual bait [5, 6]. When the feeding amount is too less, the growth rate will became slow and the breeding cycle will be prolong, resulting in the decrease of unit catches [7]. On the other hand, there are many factors exists that can be affect the feeding rate of fish, including environmental factors, physiological factors, nutritional factors and management factors, etc. [4]. These factors lead to uncertainty in the amount of feed required offish, and it is almost impossible to obtain accurate quantity by calculation [8]. The operator is often difficult to accurately grasp the most suitable feeding level to meet the needs of fish. Therefore, it is urgent to achieve automatic control of feeding process in aquaculture [9, 10].

The traditional automatic feeder can only fed a fixed amount at the setting time according to the preset program. And it is only a substitute for throwing bait by manual. It does not make corresponding changes in feeding amount and time according to the fish appetite. In fact, it does not achieve the automatic feeding [11].

The deep understanding of fish behavior shows that although there are many factors affecting the feed intake, all the factors can be reflected by their behavioral changes. In the feeding process, the speed and direction of fish will be changed at different stages [12,13,14]. In recent years, according to the feeding behavior change, the feeding amount can be adjusted has become a new research focus. As the continuous development of machine vision technology, because of its advantages of non-contact, low cost and simple development, it has become the first choice for fish feeding behavior detection and analysis [15,16,17,18,19]. As in most fish farms exists many issues such as high humidity, wide range of temperature changes, the traditional PC are not suitable for such occasions. While the embedded control technology is very suitable for such abominable environment. The feeding process can be adjusted and fed back by various feeding behavior detection sensors in real time, and make they has the ability of automatically determine the feed demand of the fish. It is considered as an intelligent feeding system with high adaptability.

In this paper, a feeding controller was designed on the basis of the existing feeding machine, and the original structure of feeding machine was not modified. The controlling of feeding amount, time and frequency of the existing feeding machine were achieved through a variety of interfaces and protocols. The feeding behavior of fish was analyzed by machine vision and embedded control technology, and the feed rate and the starting or stop** time of feeding machine were adjusted in real time. The intelligent control of feeding process was realized to reduce feed consumption and water pollution.

2 Materials and Methods

Embedded vision has mostly been based on DSP or FPGA platforms over the past 20 years. However, there are many problems. The DSP usually has low computing speed. Therefore, it is necessary to design special DSP to achieve complex operations with multi-cores, parallel processing mechanisms, or optimize algorithms for special platforms. Therefore, the cost of develo** visual application modules based on the DSP platform is higher. In addition, DSP and FPGA are suitable for parallel operations. Statements containing jump instructions are inefficient, such as “if… else…”.

With the development of ARM technology, ARM platform has brought unprecedented opportunities for the development and application of embedded visual application module. It does not need quadratic optimizations, and only needs to write efficient image processing algorithms on the PC platform. After cross compiling, we can get a visual image processing algorithm which can run efficiently on the ARM platform. Through the support of QT, the processed image and target information are displayed in the form of visual interface, so it has good interaction.

The feeding controller system designed in this paper was based on embedded Linux OS and the ARM Cortex A9 kernel micro-controller. Among them, the main control module of the feeding controller is the key part of the design. The controller collects the feeding image of fish in the breeding area via the industrial camera. And the collected images were analyzed and processed. According to the processing result, the feeding controller sends control instructions to the feeding machine to realize automatic feeding according to the requirement. The automatic feeding controller designed in this paper was shown in Fig. 1.

Fig. 1.
figure 1

The schematic diagram of system

Full size image

2.1 Hardware Platform Design

The main control chip in this paper was the i.MX6 Quad, it is an important product of NXP Semiconductor in image and video processor. It has the lowest power consumption on the basis of providing high performance processing capability. It is implemented by four 1 GHz ARM™ Cortex-A9 core processors. I.MX 6 series can provide eye-catching multimedia performance. And its high image processing capabilities can support high-definition video operation. They include 2D and 3D graphics processors; Three hardware accelerated graphics processing units (GPU); 3D 1080p video processing, support full duplex video image processing in multiple formats without CPU resources. Each processor provides interfaces for connecting to peripherals devices, such as WLAN, Ethernet, etc. [20].

The embedded hardware platform of this topic also includes the interface circuit of feeding machine (Ethernet, USB, IO, UART), human computer interface (mouse, monitor, camera), storage interfaces (SD, SDRAM, NORFLASH, NANDFLASH) and minimal system circuits (JTAG, RTC, etc.). The hardware design of the system was shown in Fig. 2.

Fig. 2.
figure 2

The schematic diagram of system hardware.

Full size image

Design of Image Acquisition Circuit.

QSC200 industrial camera was used to image acquisition, 1/3 “CMOS image sensor, video resolution 640 * 480 (15fps), 6–12 mm manual zoom industrial lens, angle of view: 53°–28°. The industrial camera was connected to the main controller through the USB interface. It can collect the image and transfer it back to the controller for processing.

Design of Communication Interface Circuit for Feeding Machine.

The interface of the feeding controller is the key part of this study, which is related to whether the feeding controller can be used normally. Through the investigation of most feeding machines in the market, the reserved or available interfaces are basically IO port, UART, Ethernet interface and so on. In order to be compatible with the feeder, it is necessary to take into account all the interfaces above, and to set aside interfaces such as WLAN, 4G and so on. The two UART interfaces use the MAX3232 to realize the level conversion of the controller and feeder. The Ethernet interface was implemented by RTL8211E-VB-CG chip, which provides faster and smoother speed for onboard Gigabit Ethernet. The USB controller was used to implement the USB interface. The WLAN interface was implemented through the S500M module.

2.2 Software Platform Design

In addition, the current embedded operating system such as Linux, which can support complex data processing, is the key to real-time decision-making [21, 22] or tracking system [23]. Therefore, it is very suitable for the development of mobile devices [24].

The design of the feeding system software is mainly based on the Linux OS platform. Before writing the acquisition, display, transmission and identification program of feeding image, it is necessary to build a programming environment. In this paper, Qt image interface system was used to develop visualization and operation interface of feeding controller. Image processing vision library Open CV was also cross platform. In order to solve the problem of compatibility problem when the software is transplanted, we must build a cross platform development environment as follows.

Development environment build. Develo** ARM-Linux image processing pro-grams requires three systems as shown in Fig. 3. Windows OS is mainly used for the preparation of program code. The PC-Linux virtual machine OS is primarily used to cross compile the written program code; the ARM-Linux OS is used to run compiled executable files.

Fig. 3.
figure 3

Development environment build

Full size image

After building Windows OS, PC-Linux virtual machine OS and ARM-Linux OS, the develo** of embedded applications can be started. The complete embedded program cross compiling and develo** process was shown in Fig. 4.

Fig. 4.
figure 4

Embedded program cross compiling development process

Full size image

Embedded Image Processing Visual Library (Open CV).

OpenCV, like Qt, is also cross platform and supports both Windows and Linux, which makes the development process a lot of convenience [21]. The flowchart of the library based on Open CV was shown in Fig. 5.

Fig. 5.
figure 5

The flowchart of the processing library

Full size image

2.3 Image Processing

The extraction method of feeding behavior index used in this paper are referred to literatures [22] and [23]. The fish target was extracted by using some image processing algorithms, such as background extraction [24]. During the feeding process, there is often overlap due to the fish activities. In order to solve this problem, this paper uses the watershed algorithm for the segmentation of overlap** targets [25, 26], and gets the image of the individual fish. Then the non-fish targets were filtered out, and the big or small targets were filtered out to get the binary image of the fish. In this paper, the position of the fish was expressed by the centroid. And the calculation of centroid was realized by the order moments [27].

Finally, the method in literature [23] was used to extract behavioral indicators during fish feeding. By using the Delaunay Triangulation, the behavior index FIFFB can be obtain [28, 29].

3 Results and Discussion

The experiment was performed at **aotangshan National Experiment Station for Precision Agriculture, Bei**g, China. The Cyprinus carpio var. specularis was selected as the subjects, which had been kept in the tank for a month and thought they had been adapted to the current environment. After the experiment, they were fed with commercial feed. In each feeding, the feed was supplied until there are more pellets floating on the water, and most fish do not eat the feed and spread out, then stop feeding and remove the remaining pellets. During feeding, the feeding image of fish were collected, analyzed and processed.

Figure 6a is a binary image of fish feeding activity after image processing, and the centroid represents the spatial distribution of fish. And Fig. 6b is the process of Delaunay Triangulation. After triangulating the centroid of the fish in each frame of the feeding process, the FIFFB of the whole feeding process can be obtained, as shown in Fig. 7. Results shown that after feeding (t > 10), the change of FIFFB is obvious, and shows a certain change rule. When the fish snatched violently, the FIFFB becomes smaller, and vice versa. After many trials and manual observation, the threshold for stop feeding was chosen as: FIFFB = 595. Meanwhile, the controller sends stop feeding instructions to the feed machine.

Fig. 6.
figure 6

Delaunay triangulation

Full size image
Fig. 7.
figure 7

FIFFB change and threshold selection.

Full size image

In order to verify the feeding decision accuracy and performance of this threshold, the decision output results were compared with the results observed by humans. The basis for manual observation is that: when the fish has almost no reaction to the feed and swims around, stop feeding, otherwise, continue feeding.

In the experiment, ten feedings were performed. When the stop feeding decision was made by expert observation, the value of FIFFB was calculated at this time, as shown in Table 1. In order to evaluate the consistency between threshold and artificial judgement, the deviation degree was used to evaluate its performance, which can be defined as:

Table 1. The FIFFB value of the 10 times feeding
Full size table
$$ D = \frac{{|{\text{A}} - {\text{X}}|}}{\text{A}} $$
(1)

Where A is the target dat, X is actual. It can be seen that the average value is 593.5, and the maximum deviation is 2.5%, which is very close to the threshold, indicating that the consistency of the two data sets is better. Therefore, it can be used as a substitute for artificial feeding.

4 Conclusion

In this paper, a feeding controller for cultured fish based on computer vision and feeding behavior was designed and implemented. At the same time, the hardware and software platform of feeding controller based on I.MX6Q microcontroller and embedded Linux OS were completed. The feeding images can be acquired and analyzed in real time. The feeding behavior of fish was quantified by Delaunay Triangulation, and the threshold of stop feeding was obtained. Compared with the traditional feeding machine, the feeding controller designed in this paper can realize the automatic intelligent control in the feeding process of fish, which plays an important role in improving the economic efficiency and automation level in the process of raising fish. However, only the Cyprinus carpio var. specularis was used in the experiment. The next step will be to validate the controller with other species to apply more varieties.