Development of an open-source Deep Learning library in REBECCA and its application in defect detection for underwater structures

AI independency for Europe

The current edge AI framework market is dominated by products like “TensorFlow”, “PyTorch” or “ONNX Runtime”, developed by companies located outside of Europe (Google, Microsoft and Meta respectively). The full potential of these frameworks is only achieved by deploying them on GPU-powered systems, but they also offer reduced versions suited for embedded systems with less computational resources. Nevertheless, the architectures supported by these frameworks are usually limited to ARM and other proprietary systems.

One of the objectives in REBECCA KDT JU is to enhance the European autonomy and sovereignty over Edge HW and SW by using RISC-V (an open, royalty free, CPU architecture). With this purpose in mind, the availability of European free and open-source AI libraries is crucial to facilitate such adaptation. To that end, Solver Machine Learning contribution to REBECCA is the adaptation and improvement of the European Distributed Deep Learning (EDDL) library to run on RISC-V processors and cover the needs of some of the REBECCA use cases.

EDDL’s role in REBECCA

As part of the REBECCA project, SML has worked on developing extension of and adapting the EDDL to support its deployment over RISC-V architectures and, by the end of the project, it will leverage the computational acceleration provided by the hardware accelerators developed in parallel by other project partners. In addition, fine-tunning efforts have been made to optimize the library software to leverage the capabilities of the REBECCA chiplets hardware and increase efficiency of CPU inference executions. With this, the EDDL library will provide an open-source AI framework that can be integrated into the project’s Use Cases.

Since AI models are in constant evolution, with new architectures and methodologies appearing each day, the EDDL has been updated in REBECCA to keep up with the latest development of the YOLO object detection model family. The library was reworked to add support for the new neural network layers required by the YOLO models, while maintaining backwards compatibility with previous iterations of layers whose standards have been updated in the last year.

To allow for an easy sharing of AI models, the EDDL makes use of ONNX (Open Neural Network Exchange), an open-source standard designed for representing machine learning algorithms such as Neural Network model. The use of this standard allows the EDDL to easily import trained models into Edge devices. By developing an in-house model serialization protocol for EDDL, Solver has been able to remove third party dependencies from big corporations (e.g. Google’s Protobuf library); which allows the EDDL to be used freely without licensing issues.

Demonstration of EDDL’s capabilities

As part of UC4 software stack, the EDDL library will execute the inference of YOLOv8n models, focusing on the real-time detection of cracks and fissures in underwater structures during jetty inspections. These models will employ images captured by RGB cameras mounted on underwater drones as input and will output the location of defects detected.

The above diagram summarizes the processes for:

Training UC4 defect detection models

Compiling EDDL using REBECCA’s SDK

Compiling a System Image using REBECCA’s SDK and integrating it into the REBECCA platform.

The diagram showcases how the EDDL can be executed as a standalone program in the REBECCA platform, but other partners will contribute to UC4 software stack. In the final version of the use case, the EDDL executables will be integrated in a partition of FENTISS XNG Hypervisor, which will allow the execution of the inference to be isolated from other processes executing in parallel on the REBECCA platform.