3D fish tracking system
Quick summary
Mitigating reduced annual flow and low flow measures
Date: 1973
Developed by:
Suitable for the following measures:
Introduction
Laboratory investigations with live-fish, i.e. so-called etho-hydraulic tests, serve to understand interactions between the hydraulics of fish protection technologies and fish behaviour and hence to improve the current design of fish passages or develop new technologies. For laboratory application, VAW of ETH Zurich developed a three dimensional (3D) fish tracking system consisting of synchronous vertically submerged cameras and a MATLAB-based 3D tracking software to determine fish locations in the flow from the recorded videos (Figure 1).
Typically, the behaviour of aquatic fauna is documented by manual protocol written down by biologists and (optional) supplementary video recording. The main drawbacks of both techniques are (i) time consuming, (ii) low time resolution, (iii) low spatial resolution, and (iv) providing only qualitative information. The present 3D fish tracking system overcomes such drawbacks by automatically and accurately providing 3D swimming tracks on a larger metric space at a milliseconds time resolution.
The software tracks several fish in 3D. Swimming path-time diagrams give a distinct ‘big picture’ of the fish movement, which helps to identify fish preferred and disliked regions. Furthermore, detailed 3D path analyses of fish interactions and fish velocities are provided as well. The details of the system are documented below. Although the 3D fish tracking system is developed for laboratory use, it may be applied in an adapted version in the field to monitor fish movements or counting, as long as the visual observation is not compromised by turbidity.
Application
Within the scope of FIThydro, VAW investigates two types of fish guidance structures (FGS), namely with horizontal (Figure 12) and vertical curved bars. These FGSs are tested with four different fish species under various hydraulic conditions to evaluate their fish guidance efficiencies and to understand fish behaviour. To this end, the 3D fish tracking system is further developed and tested in these etho-hdyraulic (live-fish) investigations. The present system is similar to that currently used by the German Federal Institute for Hydraulic Engineering (BAW) in Karlsruhe together with the German Federal Institute of Hydrology (BfG, 2018; Detert et al., 2018).
The present system consists of up to five cameras arranged in a streamwise series facing vertically upwards through the water surface, each with a distance of 1.5 m (Figure 13). Model acA2000-50gmNIR cameras from Basler are used and equipped with a 185° fisheye lens of FE185C086HA-1 (Fujifilm) (Figure 13a). The camera resolution is 3 MPx. Each camera and lens are waterproofed using a housing from Autovimation (Figure 13b). A GigE Vision 2.0 network with a Precision Time Protocol (PTP) IEEE1588 provided synchronous measurements with frame rates kept constant at 20 fps (Figure 13c). For larger control volume and longer areas, the actual system including the network switch and the high performance PC can theoretically be equipped with up to 48 cameras. However, the frame rate will be lower then.
An adapted software by Fujifilm Switzerland is used to set-up cameras and record videos. The etho-hydraulic flume is illuminated with 7x1000 W halogen lamps (Figure 12, right). Calibration of the system is essential and made in three steps: finding intrinsic and extrinsic parameters for each of the five cameras using a checkboard, calibrating five stereo cameras according to the overlapping views of camera pairs, and finally performing a rigid transformation of all stereo camera pairs to a global flume coordinate system (Figure 14a, Detert et al., 2018).
3D fish tracking is based on the detection of moving fish in each frame and associating the detections corresponding to the same fish over time. These are done by using a background subtraction algorithm and a Karman filter in MATLAB (Detert et al., 2018). The primary results of motion-based tracking are tracks in a distorted and uncalibrated 2D image frame coordinate system for each camera. Figure 14a and c show the three detected fish and noises caused by reflections from the glass window and their 2D tracks over time. After undistorting such frames and stereo calibrating the cameras, the 2D fish tracks are transferred to a 3D metric-space according to their epipolar geometry based on the camera parameters derived from the calibration (Figure 15).
The etho-hydraulic tests were done for a flow depth of 90 cm, flume width of 150 cm, distance of 150 cm between the cameras and average flow velocities up to 0.7 m/s. Under such conditions, the 3D fish tracking system provided fish positions in 3D with an accuracy of about ±5 cm and 20 fps. The challenges for a successful implementation of the system are: assignment of individual fish to the tracks, constant illumination of the flow, camera distortion, air bubbles and suspended sediment and humid conditions for the cameras. Overall, despite some shortcomings such as noise due to reflections from the glass windows, the 3D fish tracking system works well, provides important information on fish behaviour affected by fish guidance structures and has the potential for further etho-hydraulic studies.
Other information
The total costs of the present system is approx. 40’000 USD=35’000 € including camera set-up and recoding software. For current costs of the equipment, we recommend to ask the corresponding supplier listed below. Note that a cheaper camera and lens set-up can significantly reduce the total cost of the system. The MATLAB-based 3D tracking code developed by VAW will be freely available.
Links to the suppliers of equipment:
Software for 3D fish tracking:
- Available on request.
Relevant literature
- BfG (German Federal Institute of Hydrology), 2018. The behaviour of fish in fishways – BfG and BAW’s ethohydraulic tests. Annual report of 2016/2017, pp.43. https://doi.org/10.5675/bfg-jahresbericht_2016/2017.
- Detert, M., Schütz, C., Czerny, R. (2018). Development and test of a 3D fish-tracking videometry system for an experimenal flume. In Proc. River Flow 2018 - Ninth International Conference on Fluvial Hydraulics, E3S Web of Conferences 40: 03018. https://doi.org/10.1051/e3sconf/20184003018
