Antonio Sgorbissa is full Professor at the Università degli Studi of Genova. He is the director of the RICE lab, Robots and Intelligent systems for Citizens and the Environment.
A Brief Introduction to Aerial Drones for Photovoltaic Plant Inspection
Aerial drones, particularly quadcopters, have become extremely popular in recent decades. From recreational use to aerial videography of sports events, from package delivery to large-scale mapping for inspection or agricultural purposes, drones offer a unique ability to « view things » from above at a significantly reduced cost. We’ve all seen them flying over beaches or the countryside – though less often in cities due to flight regulations – their four propellers spinning at high speed and producing a distinctive noise.
One area where drones can play a crucial role is in the inspection and monitoring of large photovoltaic (PV) plants for detecting defects in PV panels. This aligns with the goals of the SOLARIS project, which seeks to « make the most of every ray ». Various sensor technologies are already available for this purpose, ranging from widely used thermal cameras that capture temperature inconsistencies across surfaces (a clear indicator of defects) to innovative sensors and algorithms leveraging Deep Learning technologies to accurately recognise and potentially segment damaged areas. Some of these advancements are being developed within the SOLARIS project, and you will hear more about them soon!
Although quadcopter technology is widely used, PV plant inspections are still primarily conducted manually. This means an expert operator controls the drone with a remote controller – an approach that may be fun at first, but what happens when the same operations need to be repeated multiple times? A solution is to automate the drone’s flight, with the drone following a flight plan made up of GPS waypoints. In both manual and automated scenarios, operations typically occur under VLOS (Visual Line of Sight) regulations, as BVLOS (Beyond Visual Line of Sight) operations introduce additional challenges and stricter regulations.
In automated flights, when a flight plan consists of a sequence of waypoints, the drone typically moves in a back-and-forth pattern, ensuring the entire area is covered. Each back-and-forth movement captures a « stripe » of images, which overlap with adjacent stripes to ensure comprehensive mapping of the area. This approach, though well-established, requires flying the drone at a high altitude to mitigate GPS errors that could affect the final map. However, higher altitudes result in lower image resolution. Furthermore, covering the entire area often results in capturing unnecessary images of non-PV areas, such as ground, grass, rocks, or buildings.
SOLARIS Proposal
SOLARIS proposes a more autonomous approach to inspecting PV plants. While the drone will still operate within VLOS constraints, it will do so autonomously. This means the drone will make decisions based on sensor inputs and the assigned task to achieve its goal most efficiently. In the SOLARIS project, drones will follow rows of PV panels from one end to the other, using onboard sensors and AI algorithms to autonomously navigate from one row to the next. This process, known as « visual servoing, » relies on visual inputs for navigation, much like an intelligent being would use their eyes, instead of simply following GPS waypoints as in traditional flight plans. While doing this, the drones will carry the necessary sensors for inspection, including thermal cameras as the baseline solution, along with other potentially groundbreaking sensors developed during the SOLARIS project.
But that’s not all! Traditional flight plans consist of a sequence of GPS waypoints computed from the borders of the plant’s polygon, without accounting for how the PV rows are distributed in the field. This approach is suboptimal since it does not consider the path’s energy efficiency or minimise the flight distance. In SOLARIS, however, we plan to change this. By using an aerial image of the entire plant (captured by satellite or an initial drone flight), we are developing algorithms to segment PV panels from the background. These algorithms will then compute the most efficient path to cover the entire area, considering the unique arrangement of the PV panels. Our researchers are exploring optimisation algorithms, from those that provide the optimal solution to heuristic ones that deliver a suboptimal but faster solution, enabling large plants to be processed in the blink of an eye.
An interesting aspect of this challenge is that, under certain conditions, minimising path length or energy consumption is not the only priority. Visual servoing is effective when the drone is moving along a panel row, as the camera provides a consistent reference. However, when jumping from one row to another, the drone relies on GPS input, which can be prone to errors. This is especially true when flying at low altitudes: even a small GPS error (1-2 meters) can cause the drone to miss the start of the next row: consider a drone that reaches a position with coordinates X, Y, but the start of the PV panel is located at position X+2 meters, Y+1 meter. The drone might miss it!
Since long flight segments using GPS only are dangerous, we propose a strategy that minimises the length of flight segments that are performed while “jumping” from a PV row to the next one, which obviously cannot be done with visual servoing. Suppose that this strategy produces a longer path: while a longer path may initially seem less efficient, it could be beneficial if it reduces the distance flown without visual servoing. In other words, minimising « jumps » between PV panel rows could improve overall inspection accuracy.
Where Are We Now?
We are almost ready to release the first version of our system, capable of:
- Computing the best path based on aerial imagery.
- Autonomously flying along the chosen path using visual servoing techniques.
The image below shows a simulated drone flying over an aerial snapshot of the DTU plant. The green path has been automatically computed by our system to find the most efficient route. The box in the top left shows what the aerial drone’s camera sees, which serves as the basis for navigating along the PV panel rows through visual servoing. While it may look like we are ready to deploy this system on real drones flying over real PV plants, as is often the case in research, new challenges continue to emerge.
How do we handle larger plants that cannot be covered with just one battery charge and require drones to return to a recharging station? How do we manage multiple drones covering the plant simultaneously while avoiding collisions and other conflicts? How do we adapt visual servoing to situations where PV panels are not on flat terrain, but rather follow the sloping profile of a hill, requiring adjustments in flight altitude?
We already have answers to these challenges, but we haven’t tested them yet. Stay tuned for updates!
French