BOOK A DEMO
Teaching AI to See Heat: How Transformers Are Learning to Find Solar Panel Faults

Teaching AI to See Heat: How Transformers Are Learning to Find Solar Panel Faults

The paper that finally proved AI can “understand” why a solar panel is broken — not just recognize that it is.

If you've ever watched a solar farm being inspected, you know the drill: a drone flies over the rows of panels with a thermal camera, capturing thousands of infrared images. Later, someone stares at those images — or runs them through a model — looking for the bright orange spots that mean trouble.

Here's the uncomfortable secret: the AI models that do this screening are usually "black boxes." They can tell you a panel is faulty, but nobody can say why the model thinks so. Is it actually looking at the hotspot? Or is it noticing something irrelevant, like a shadow pattern or a camera artifact? For a utility-scale solar operator deciding whether to send a crew to replace a panel, that ambiguity is a dealbreaker.

A new paper by Serra Aksoy (LMU Munich) takes aim at exactly this problem — and in doing so, it may have unlocked the trust barrier that has kept AI out of solar inspection rooms for years.

The Big Idea: Don't Just Ask What — Ask Why

The study, published in September 2025, did something no one had done systematically before: it benchmarked the four most popular deep learning architectures on thermal solar panel images, then used an explainability technique called XRAI to check whether the models were making decisions based on real physics.

Think of XRAI as a highlighter. When a model looks at a thermal image and says "this panel has a diode failure," XRAI draws a heatmap showing exactly which pixels drove that decision. The question Aksoy asked was simple: do the highlighted pixels match what a thermal physics textbook would predict?

ModelBinary Accuracy (faulty vs. healthy)Multiclass Accuracy (11 fault types)
ResNet-18 (CNN)93%69%
EfficientNet-B0 (CNN)93%69%
ViT-Tiny (Transformer)94%72%
Swin-Tiny (Transformer)94%73%

The dataset: 20,000 infrared images spanning normal operation plus 11 fault categories — cell defects, hotspots, diode failures, cracking, soiling, vegetation shading, offline modules, and multi-occurrence versions of several.

What the Models Actually Learned

Here's where it gets genuinely interesting. The XRAI analysis revealed that the models weren't cheating. They had learned real thermal physics:

  • Cell defects → the model focuses on a round, localized hotspot — exactly where a damaged cell would overheat.
  • Diode failures → the model looks at a long, linear hot strip along the panel edge — precisely the thermal signature of a failing bypass diode.
  • Vegetation shading → the model tracks the thermal boundary between the shaded and unshaded part of the panel.
  • Cracks → attention follows the linear path of the crack, showing the model understands how fractures redistribute heat.

This is a big deal. It means the models aren't finding spurious correlations that could break down on a cloudy Tuesday in another climate. They've essentially reverse-engineered the failure physics from images alone.

The Catch: What AI Still Can't See

Not all faults are created equal. The study found a dramatic performance split:

  • Electrical faults (diode failures, offline modules): F1-scores above 0.90 — excellent.
  • Soiling (dust and dirt buildup): F1-scores of just 0.20–0.33 — the worst class by far.

The reason is humbling: a layer of dust barely changes a panel's temperature signature. In a desert installation, where soiling can cut output by 20–40% per year, thermal imaging alone simply cannot see the problem. The paper's authors argue this justifies multi-modal inspection — pairing thermal with optical imaging, electrical measurements, or impedance testing.

There's also an economic argument buried in the data that every solar asset manager should know: analysis of operational plants shows electrical and transformer failures account for 68% of all failure-related energy losses — despite being only 6.5% of all failures. Getting electrical fault detection right matters more, per dollar of lost energy, than chasing every other failure mode combined.

Why This Matters for Solar Due Diligence

For anyone doing technical due diligence on utility-scale solar assets — whether pre-acquisition, pre-construction, or during periodic performance audits — this paper offers a practical upgrade path:

  1. Explainable screening = defensible reports. When you can show a lender or buyer why the AI flagged 47 panels with diode faults (with attention maps overlaid on thermal images), the inspection report stops being a black box and becomes evidence.
  2. Trustworthy automation. Human experts trust tools they can verify. Physics-validated AI is the first step toward automated drone inspection that operators actually rely on — not just for the screeners to review.
  3. Prioritize what matters. The data says electrical faults cost the most per incident. The models detect them best. That's a perfect alignment between what's economically important and what's technically feasible — so deploy detection budgets accordingly.

The Bottom Line

This paper's real contribution is a template: validate AI decisions against physics, not just against accuracy numbers. Swin Transformer (94% binary / 73% multiclass accuracy) is now the best-documented architecture for thermal PV fault detection — and, more importantly, we now know its reasoning can be trusted because it aligns with how heat actually behaves in a broken panel.

The next frontier is obvious: multimodal sensing to close the soiling blind spot, and physics-informed architectures that embed thermal equations directly into the network. For solar farm owners, the message is encouraging — AI isn't just getting more accurate. It's getting explainable. And that's what makes it deployable.

Antevolt Labs

Get new research the day it drops

Benchmarks, standards and deep dives from the plants we inspect — delivered to your inbox. No fluff, no sales pitch, just findings.

One email when we publish. Unsubscribe anytime.

Related articles

Taming a Gigawatt: What It Really Takes to Plug a Giant Solar Farm Into the Grid
Grid & interconnection

Taming a Gigawatt: What It Really Takes to Plug a Giant Solar Farm Into the Grid

JUL 20, 2025
The Digital Twin That Embraces Uncertainty
Site & ground

The Digital Twin That Embraces Uncertainty

DEC 12, 2024
Five Solar Farms That Know the Whole World
Asset performance

Five Solar Farms That Know the Whole World

JUL 25, 2024
Solar Panels That Teach Themselves: The Rise of Self-Aware PV
Detection accuracy

Solar Panels That Teach Themselves: The Rise of Self-Aware PV

JUL 10, 2026
When the Grid Says “Slow Down,” the Panels Feed the Crops
Agrivoltaics

When the Grid Says “Slow Down,” the Panels Feed the Crops

JUL 10, 2026
The Solar Panels That Negotiate With Crops
Agrivoltaics

The Solar Panels That Negotiate With Crops

MAR 23, 2026
Forty Years of Solar Fire Research, Mapped
Safety & compliance

Forty Years of Solar Fire Research, Mapped

DEC 16, 2025
The Performance Ratio Paradox: Why Your Array’s Biggest Enemy Is Human
Asset performance

The Performance Ratio Paradox: Why Your Array’s Biggest Enemy Is Human

NOV 20, 2025
Can an AI See a Wound in the Soil?
Site & ground

Can an AI See a Wound in the Soil?

NOV 17, 2025