A three-dimensional marine plastic litter real-time detection embedded system based on deep learning☆

Introduction：

With the improvement of human living standards, the plastic in- dustry has become one of the world’s largest manufacturing industries (Liu et al., 2023), and plastic litter has proliferated in the world. Due to both natural processes and human activities, millions of tons of litter flow into marine, with over 80 % of marine litter consisting of plastic litter (Schwarz et al., 2019), thereby directly impacting ecological en- vironments. Projections suggest that by 2040, the volume of plastic pollution entering the oceans may nearly triple, resulting in an annual increase of 23 to 37 million metric tons (McGlade et al., 2021). This alarming increase of marine plastic pollution poses a significant threat to marine life and ecosystems, rendering it an essential and irreversible global challenge. Consequently, more and more attention has been paid to the research on marine plastic litter problems (Turrell, 2018; Parker- Jurd et al., 2022).
Currently, various remote sensing platforms are utilized for long- time monitoring of marine plastic litter, primarily employing airborne (Veerasingam et al., 2022; Garcia-Garin et al., 2020; Balsi et al., 2021)

and spaceborne (Basu et al., 2021; Kremezi et al., 2022; Taggio et al., 2022) technologies. However, these platforms lack integrated litter removal capabilities, and their effectiveness is compromised by weather conditions. In addition, existing methods are inadequate for addressing underwater plastic litter because acoustic waves are the only medium capable of transmitting in underwater complex environments. The challenge posed by underwater environments can be summarized as follows: acoustic propagation characteristics (Yang et al., 2024a; Yang et al., 2024b), limited energy availability of devices (Yang et al., 2023a; Yang et al., 2023b), and size constraints (Jiang et al., 2022). Image quality is significantly impacted by harsh underwater environments, which include factors such as seawater interference and scattering caused by suspended particles, as well as the differential attenuation of various light wavelengths. These elements contribute to the degradation and distortion of underwater images, resulting in issues such as blurri- ness, low contrast, color distortion, and severe haze (Wang et al., 2019). Such shortcomings destroy complete image information, thereby reducing the accuracy of target detection. On the other hand, although there is extensive image data on marine litter, for example the

AquaTrash datasets (Harsh Panwar et al., 2020), they primarily focused on coast and sea surface litter. There is a notable scarcity of underwater litter datasets due to the challenges associated with underwater data acquisition.
A review of the existing literature (Veettil et al., 2022) reveals that, due to the aforementioned challenges, the majority of studies concen- trate on the detection and mapping of marine plastic litter, primarily focusing on either coastal or floating objects. Consequently, underwater litter has received significantly less attention in research. Traditionally, marine plastic litter is addressed through methods such as fishing nets, salvage, and litter cleaning vessels. Although the first two methods are relatively straightforward to implement, they require substantial manpower and pose safety risks. Conversely, litter cleaning vessels are highly efficient, but they need considerable cost, rendering them impractical for widespread application. Therefore, there is a pressing requirement for low-consumption and high-efficiency autonomous un- derwater vehicles (AUV) equipped with capabilities for recognizing marine plastic litter to enhance marine environmental protection.
Deep learning technologies have been widely investigated on image recognition, owing to their remarkable ability to identify and under- stand features and patterns present in large image datasets. This capa- bility is particularly significant for the recognition of marine litter (Abdelaadim Khriss et al., 2024). Numerous research efforts have concentrated on the application of deep learning for marine litter detection (Gonçalves et al., 2020; Politikos et al., 2021; Hidaka et al., 2022; Kikaki et al., 2024; Zhang et al., 2024; Nguyen and Dang, 2024; Zhao et al., 2024). For instance, Ref. (Zhao et al., 2024) introduced the aerial-aquatic speedy scanner, which integrated super-resolution reconstruction with an enhanced YOLOv8 detection network. The experimental results demonstrate the efficacy of this method in detect- ing underwater litter. Nevertheless, despite the superior recognition capabilities of many high-performance deep learning technologies, their real-time implementation on embedded systems in real-world applica- tions requires careful consideration of computational complexity.
With the rapid advancement in intelligent systems, deep learning technologies have been successfully applied in various underwater application platforms (Valdenegro-Toro, 2016; Valdenegro-Toro, 2019; Watanabe et al., 2019; Yi-Chia et al., 2020; Escobar-Sa,nchez et al.,2022;
Wang et al., 2022; Sa,nchez-Ferrer et al., 2023; iljeg et al., 2023; Fulton
et al., 2019), including remotely operated vehicles (ROV) and AUV. For instance, focusing on coastal litter, an AUV utilizing convolutional neural networks has been explored to classify marine litter by using forward-looking sonar (Valdenegro-Toro, 2016). The proposed method achieved an accuracy of 80.8 % for binary detection and 70.8 % for multiclass detection. A ROV referred to as WASSP S3 has been developed for the purpose of mapping extensive accumulations of marine debris in
shallow water, as indicated in (iljeg et al., 2023). However, it is
important to note that this ROV is not equipped to detect marine litter that is smaller than 1 m in size. It can be seen that most existing systems are limited to single-scene applications, which constrains the three- dimensional detection of marine plastic litter. Furthermore, the detec- tion models are trained by previously damaged practical underwater image datasets, which adversely affects model performance.
To overcome these challenges, a three-dimensional marine plastic litter real-time detection (3D-MPLRD) embedded system based on deep learning is proposed in this article. The 3D indicates that our proposed system focuses on not only coast and sea surface, but also underwater environments. Specifically, a newly constructed marine litter dataset is introduced, which is supplemented with various image processing methods to enhance its comprehensiveness. Additionally, to mitigate the issues associated with damaged underwater images, image enhance- ment technologies are employed to improve the quality of the training dataset. Subsequently, the proposed system trains a deep-learning based detection model using the enhanced dataset. The trained model is then compressed and quantified for deployment on embedded devices. Finally, practical experimental results show that the implemented 3D-

MPLRD embedded system achieves satisfactory performance.
The contributions of this article are summarized as follows: [1)].
1) To overcome the 3D multi-scenario limitation of embedded systems for marine plastic litter real-time detection, a 3D-MPLRD embedded system is proposed and implemented by incorporating deep learning models. It can provide a foundational reference for marine envi- ronmental protection based on intelligence systems, rather than traditional manual methods.
2) To improve the performance of the deep learning model in the 3D- MPLRD embedded system, we constructed a marine litter dataset. This dataset integrates public datasets with newly captured marine plastic litter images and utilized additional methods to enhance the comprehensiveness of the dataset.
3) Traditional training methods face several image damage caused by harsh underwater environments, hence, image enhancement tech- nologies and image assessment methods are adopted to optimize the low-quality images in the training dataset. Furthermore, the images that are not enhanced can be input directly to trained models without image enhancement in real-time embedded systems.
4) To implement the real-time detection, the trained detection model is compressed and quantified for deployment on embedded devices. Practical experimental results indicate that the implemented 3D- MPLRD system performs effectively, offering a reliable approach for integrating deep learning technologies into embedded systems.
The remainder sections of this article are organized as follows. In Section II, the detailed elaboration on the proposed 3D-MPLRD system is provided, including the overall framework, flowchart, constructed ma- rine litter dataset, image quality assessment, and embedded imple- mentation. Section III presents the results and analysis of the detection of practical datasets. Finally, the conclusions of this article are drawn in Section V.
Notaions: Bold uppercase and lowercase letters denote matrices and vectors respectively. The notation Ψ specially denotes a set, and notation Ψn denotes the n-th element of the set Ψ . Superscript (.)T denotes the transpose. Notation 聂 .聂1 and 聂 .聂2 denote l1 and l2 norms respectively. Notation max(a, b) denotes the maximum between a and b, and max (a) denotes the maximum of a. Similarly, notation min(a, b) denotes the minimum between a and b, and min (a) denotes the minimum of a. Notation ρ(a, b) denotes Euclidean distance between a and b. Notations ⊗ , ⊕, and Θ denote the parameterized logarithmic image processing (PLIP) addition, subtraction, and scalar multiplication operations (Panetta et al., 2010), respectively. Notation ⊙ denotes convolution operation. Notation U and ∩ denotes the intersection and union opera- tion for sets respectively. Notation ħ (Φ) denotes the function that cal- culates the total elements of the matrix Φ . Notation〈.〉denotes round calculation.

2. Proposed 3D-MPLRD embedded system
   In this section, a detailed elaboration is provided, including the overall framework, flowchart, constructed marine plastic litter set, image enhancement, and embedded implementation of the proposed deep learning system.

2.1. Overall framework and flowchart
The flowchart of the proposed 3D-MPLRD system is illustrated in Fig. 1, which can be categorized into three main components: the establishment of a combined dataset, the creation of a trained dataset through image enhancement, and the performance evaluation based on detection results. Specifically, the first component involves generating an extended dataset through various image operations, such as mirror- ing. This extended dataset is then combined with existing public datasets

to form a comprehensive dataset. Subsequently, the combined dataset is divided into an original training set, a validation set, and a test set. The low-quality images of the original training set are improved by image enhancement technology due to harsh underwater resolution, and the final training set is utilized for training the deep-learning based detec- tion model. Finally, performance analysis is conducted in terms of various indicators based on the detection results of the test set.