Inspecting or repairing underwater pipelines is slow and risky for human divers. The goal of this project was to let an underwater Remotely Operated Vehicle (ROV) find a pipeline from its camera feed and line itself up with it automatically.

Tech stack: Python, ROS, Gazebo, OpenCV, PyTorch, U-Net, YOLO, PCA, PID control, QGroundControl, BlueROV2

How the system works

Simulation and data

A Gazebo simulation with pipelines was set up in the DAVE Aquatic Virtual Environment to generate synthetic data. The simulated pipeline images were annotated to build a custom dataset for segmentation.

Pipeline detection and pose estimation

1. Train and compare U-Net, YOLOv5 and YOLOv8 for pipeline segmentation.
2. Run the best model on each camera frame to get a segmentation mask of the pipe.
3. Apply Principal Component Analysis (PCA) on the mask to estimate the pipeline's orientation.
4. Pass the orientation to the controller as feedback.

Alignment control

A PID-based alignment framework for the simulated ROV uses the pipeline's orientation as feedback to align the vehicle with the pipe.

From simulation to a real vehicle

To test beyond simulation, a Python script using QGroundControl's API extracted live frames from a BlueROV2 in a water tank. YOLOv8 was then trained on real underwater pipe images from the tank, and segmentation and pose estimation were tested on the live feed. This also meant working on the integration between the vision software and the vehicle through physical testing.

Video