Fifth Integration week, in Lisbon

.

The BugWright2 project

Autonomous Robotic Inspection and Maintenance on Ship Hulls

Integration week in Lisbon

[Integration Week]

👷 For the next in-person Integration week, partners went to Lisbon, Portugal in September 2023.

In Lisbon, the consortium had the chance, for the first time, to get access to a large scale ship (140m) thanks to the Portuguese Navy. The objective of the week was to validate the integration of all the agents around a single infrastructure with a size representative of the final target application in shipping. Also, as a follow-up of the integration week in Porto, this was our second opportunity to bring all the agents together into the same software environment, sharing their localisation and their data in a common user interface. The second objective of the integration week in Lisbon was to demonstrate the system to external stake-holders, including the Portuguese Police and representative of a class society.

UIB team’s activities were quite bread. They continued with the development and integration of the different processes aiming at the implementation of the first stage of a BUGWRIGHT2 inspection mission. This first stage aims at building the localization framework for the rest of robots and providing a first set of inspection data (comprising flight log, images, point clouds and mesh, including the positioning information for the data collected and the corresponding processing results; all this data are available for the rest of robots). During this first stage, the inspection-oriented drone developed by the UIB operates as a standalone platform and flies in front of the structure to inspect, collecting in a fully autonomous, systematic way the inspection data. A number of processes run next in the drone ground station, in order to: (1) build the localization framework [methodology developed by the Univ. of Klagenfurt team], (2) build a mesh (to be used by the rest of robots along the BW2 inspection pipeline) and (3) detect defects in the images collected.

As part of these developments and integration efforts, a number of tests were performed on the outer hull of a real vessel for checking the suitability of the processes that lead to the generation of the localization framework [developed by the Univ. of Klagenfurt team]. The localization framework is based on visual markers (ARUCO tags) and an Ultra-Wide Band network. The rest of robots involved in a BW2 inspection mission make use of the resulting calibration to plan their motion and conclude the inspection at a more detailed level. Continuing with the developments and integration efforts, a number of tests were performed for checking the suitability of the processes leading to building the mesh of the outer hull of the vessel. The mesh, apart from being an output of an inspection mission, is intended to be useful for the rest of the robots of the inspection pipeline, as input for their motion planning modules.

Several experiments on defect detection were also conducted for the different flights performed, by processing the images collected. The output of this step comprises several files, including the images resulting from the processing, among which there is a file that indicates where the defects are suspected to be. This information can be used for planning the motion of the different robotic platforms in the second stage of a BW2 inspection mission. Part of the development/integration of the different elements involved in the first stage of the BW2 inspection pipeline took place in Porto and Lisbon, but it is intended to be continued/completed in the UIB laboratory along the next months, to be tested during the next integration meeting.

UNI-KLU’s experiments in Lisbon focused on integrating multi-robot multi-coordinate system localization in real-time. With the insights gained in the last Integration week in Porto (in June), two improvements had to be tested:

a tide-irrelevant robot initialization and an improved anchor calculation. For the first part we switched to a navigation algorithm relying only on sensors on the ship hull or the robots, thus removing the need for the tide-effected GNSS.
For the second part a new anchor calculation algorithm was tested to make use of the ship hull shape and other prior known knowledge.

Both tests showed good improvements compared to the previous IW and the final visualization with the 3D mesh was deemed to be better.

CNRS tested the crawler localization on an actual ship hull at scale using the Leica and UWB. Our team was in charge of conducting a 3D scan using a total station and employing UWB technology for localization system. We deployed both the crawler and UWB equipment on the ship, conduct structure inspection tests. During the inspection process, we gathered data such thickness measurement, path panning, localization.

Following this, we integrated the scan of the ship and mesh from UIB into both the localization system to test it, and tested the mesh for planning.

Lastly, we tested the integration of the VR system with the crawler using the ship mesh. CNRS established a network that facilitated the real-time visualization of the crawler’s data on a VR system. This VR system not only allowed for data visualization but also enabled us to control and send missions to the crawler system.

UT conducted evaluation sessions with potential end users of the user interface (UI) in BUGWRIGHT2. The UI in its current state comprises a web-based desktop interface enabling robot mission planning, monitoring and measurements visualization. The 2D and 3D visualization capabilities are further extended by a virtual reality (VR) based interface, enhancing situational understanding through offering quick way of monitoring several robots at once. During evaluation, the verbal user feedback was throughout positive. Detailed results will be made available in Deliverable 7.4.

In Lisbon, the partners from RWTH and Trier again worked together to evaluate the user interface. For this, Jan (from Trier University) and Simon (from RWTH University) setup a simulated control station to perform virtual missions and collect feedback from all tested participants. Later on Simon joined Sebastian (also from RWTH) to visualize data collected on the ship and further test the ArUco marker detection of the HoloLens2 setup.

During the integration week in Lisbon the following objectives were achieved for RINA. First objective was to attain an in depth understanding of the progress achieved by each partner in the last months and the work that is planned to be completed by the end of the project. Secondly, a quick interview was performed with each partner to collect the necessary data for the evaluation task 9.4. According to the Grant Agreement, Key Performance Indicators were defined in the beginning of the project in D1.1 and should be reviewed for the evaluation of the platforms. These Key Performance Indicators were discussed with each relevant partner and data were collected regarding the experiments performed or are planned to be performed in order to achieve these indicators. In addition, further communication was established with the partners to finalise the data collection. Furthermore, the deliverable 9.4 was discussed with the beneficiary and the work was organised. A table of content was developed, and the work was allocated to the partners that will contribute on the writing of the deliverable. An initial list of questions was defined that will be circulated to the end users that have participated on the various missions to evaluate the user acceptance of the platforms. Finally, RINA as an end user has provided input on the evaluation of the VR system.

During the 5th Integration Week in Lisbon, CETIM had the opportunity to test in real conditions on the NRP BERRIO the demonstrator CETIM has developed as part of the BUGWRIGHT2 project, based on the reconstruction of guided waves to detect and locate defects and obstacles (WP3).

The positive conclusions of these tests are :

the successful positioning of the robot on the boat’s hull, as well as the various movements carried out as part of the scans.
The acquisition of exploitable data via contactless EMAT probes.
encouraging initial results, enabling detection of the various obstacles encountered (welding, stiffeners, porthole, etc.).

Simon Ferrier and Olivier Simonin from INSA Lyon took part in the integration week in Lisbon at the shipyard. One of our objectives was to acquire real configurations of hull surfaces: the shape of the hull, the presence and shape of obstacles, and areas off-limits to robots. We collected images and data from the supply ship provided by the Portuguese navy. We also supported our partner LakesideLab (LSL) when they carried out experiments with drones. In particular, we recorded their experiments (several videos of these experiments were produced).
Throughout the week, we discussed with the partners the capabilities and limitations of their robots, with a view to incorporating them into our models/simulations and planning algorithms.

For Blueye, the Integration week event marked the transition of the project’s leadership from Borja Serra over to Petter Sjursen. We have previously developed and supplied the Blueye X3 underwater drones as a platform for the ongoing development of autonomous ship hull inspection. Since our underwater drones have already been delivered for further development in the project, our primary responsibilities during the integration week in Lisbon were providing on-site technical support and general assistance for NTNUs and UiPs autonomous ship hull inspection platforms.

During the integration week, we actively engaged in technical support tasks. Specifically, we addressed battery charging issues within a surface unit, ensuring the continuous operation of the Blueye X3 drones. Additionally, we assisted UiP in finding effective mounting solutions for their sonar setup, enhancing the overall performance of their autonomous ship hull inspecting platform. This week offered a valuable opportunity for me to acquaint myself with fellow project participants and understand how our technology contributes to the overall project goals.

The Consortium would like to thanks UPORTO and the REPmus event for their welcome, support and help during the week.

#h2020 #euproject #integrationweek #fieldtests #research #euproject #dronetechnology #innovation #testing #data #robots #augmentedreality