The BugWright2 Project
Autonomous Robotic Inspection and Maintenance on Ship Hulls
Reporting Period 1
[January 2020 – March 2021]
Our First Review Meeting took place the last week of May with the Consortium, the Project Officer and external Experts.
During this online meeting, we presented the project with its highlights so far. All WP Leaders shown their work and various demonstrations took place online.
- User requirements documented in Deliverable 1.1
- A multi-perspective approach has been followed engaging all stakeholders (class, owner, service provider) in the elicitation of requirements
- KPIs have been drafted to validate technology (measuring pilot impact, end- user satisfaction and tech components performance)
- Mission scenario specifics have been designed (in liaison with WP9)
- Intensive work in the design of the new platform to accommodate additional payloads
- During 2020, two prototypes have been designed, produced, and tested thoroughly, and a new 3rd and final prototype is about to be sent for production in 2021
- Regular discussions with service suppliers helped better understanding of the needs
- Feedbacks integrated in the R&D : IP67 / light crawler for GLM, wireless crawler for AASA
- Middleware selected at the beginning of the project: ROS as the main communication mean (aerial, crawler, visualisation). DUNE for the underwater systems because of the large body of existing work. ROS-DUNE bridge for integration (by UPORTO).
- Preliminary simulation environment provided to all partners in the first months of the project (Deliverable submitted): Combination Gazebo-ROS, expanded and improved by INSA, in use at RWTH, KLU, specific simulation for multi-robot coverage in use at INSA/LSL.
- National comparative analysis has started. At the outset, WMU has developed a questionnaire taking into account all pre-requisite elements. Jurisdictions of this deliverable includes Norway, Netherlands, US, China, Canada and Singapore. Reports for Netherlands and Norway are complete with the report for the US currently ongoing.
- Schema of a prototypical hull inspection process (i.e., task sequence, task characteristics) has been created by UT: Identification of 23 critical factors for the social acceptance of BugWright2 that refer to the six categories task, technology, visualisation, human, hybrid team, and organisational context. Evaluation of state-of-the-art robot technology with a focus on user acceptance (i.e., perceived strengths and weaknesses of technology, (interface) design, system operation, and maintenance).
- The required stages to go from the tele-operated surveys to the full-system evaluation were considered and the metrics and performance measures to evaluate the system were defined. We described the services offered (visual or ultrasonic inspection, hull cleaning) by every end-user, as well as we defined the means to evaluate them.
- From the specifications of the implementation of the large-scale pilot, we concluded to the need for the derivation of an appropriate training schedule for the robot operators and for extended training and familiarisation activities for them to be able to conduct RAS inspections.
- Organised deployment of off-the-shelf platforms to the end-user.
- GLM focused its activities on three sub-tasks: Familiarisation & Training, Class Certification and Onshore & Onboard Inspections.
- An onboard class survey of a vessel took place with the use of the BUGWRIGHT2 robotic platforms.
- With the aid of SBK, design and preparation works of the hull section mock-up have been implemented, while its construction and installation has been relocated to an appropriate location in Aspropyrgos, Greece. This mock-up will contribute significantly on the demonstration of the capabilities of the robotic means for the inspection of ships.
- MAV adaptation to BUGWRIGHT2 requirements in progress.
- Control architecture oriented towards visual inspection developed and available.
- Development of multi sensor state estimation and mapping functionalities.
- Depth data based approach: Successful results over a number of public datasets.
- Vision based approach: Being integrated within the UIB control framework
- Simulation environment to evaluate VINS algorithms.
- Photorealistic Unity3D Simulation based on FlightGoggles
- Extended to generate VR images for real drone flights
- Integration of the UNI KLU VIO module
within the UIB MAV control framework.
- Prototype localisation system integrating Crawler Odometry, IMU, UWB
- Design of a specialised localisation framework integrating the manifold constraints (Accepted at ICRA’21)
- Map the surface below the crawler: Map and segment the obstacles such as protruding objects.
- Build a textured representation of the obstacles.
- A VPN link between the CNRS and the visualisation partners in Aachen (RWTH).
- Data containers: generation of data samples containing pose, path, obstacles and their associated visualisation texture.
- Collision avoidance has been implemented based on point-cloud readings compressing into a single plane laser scan.
- Autonomous trajectory tracking using the prototype localisation system from WP4 has been implemented and field tested.
- Mission creation based on slicing a 3D mesh has been created.
- A sweeping mission has been performed in the field.
- Pioneer software interfacing with tests in the lab with the vehicle.
- Procured of DVL-A50 from Waterlinked
- Initial work on visual odometry
- Use of video recordings from Blueye and Glafcos
- Connection with UNI-KLU and UIB
- Path planning framework.
- Tests with Lamb waves and available contact piezoelectric transducers:
- Approach 1: Proof of concept of ultrasonic localisation and mapping working
- Approach 2: restricted to rectangular shapes, results are more accurate, more robust and scale to larger plate surfaces, successfully deployed on the crawler.
- Guided waves trials performed with shear wave transducers.
- Image processing for visual inspection
- Several approaches developed/under development for the two cases considered:
- detection of defects, e.g. corrosion and coating breakdown (CBC, mostly in the inner hull)
- detection of attachments on the hull, i.e. biofouling (outer hull, mostly submerged side)
- Modelling of the tethered multi-crawler inspection task
- Modelling of the MAVs visual inspection task
- Constraints modelling and problem(s) definition
- Proposal of several planning algorithms
- Analysis of algorithms performances
- A paper submitted to CP 2021
- Development of a 2.5D Simulation framework
- Proposal of several inspection algorithms
- Generation of defect for tests
- Analysis of algorithms performance
- 3D models of ship/drone/crawler/pioneer integrated from simulator into Unreal™
- Real-time rendering of model data, localisation data, and live video streams
- Live communication between Unreal™ and ROS
- Support for multiple VR hardware platforms (HMD, CAVE, Tiled Display Wall)
- Sketching of user interfaces
- First prototype of an interface for the monitoring of crawlers -> Demo
- First prototype of a VR interface for the monitoring of drones -> Demo
- Path planning for drones from within Unreal™ -> Demo
- Action plan for the UI design process and evaluations
- Personas created from the results of questionnaires and video interviews
- Preparation of design workshops for the user interface
- Decision Support for Maintenance on-going
- Creation of the website
- External and internal communication
- Dissemination (conferences, workshops, papers, etc. )
- Global market study
- Primary collection/generation of data from all partners according to the FAIR access
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grand agreement No 871260.