Case studies · Forensic walkthroughs

Real fleets. Real degradation. Caught early.

Two forensic walkthroughs on production supercomputers — replaying real hardware-degradation events through the same forward-only model. At every point in time the model sees only the data up to that moment: no hindsight. Each one shows exactly how much lead time an operator would have had.

The case studies

CINECA · Bologna, IT Tier-1 supercomputer · 4× NVIDIA V100 per node · ~32 PFLOPS

Marconi100 — an NVIDIA V100 node drifting toward 2× its power.

A real degradation event on CINECA's Marconi100, replayed across 30 months of continuous telemetry. OrionLinks flagged the failing node months before it reached its worst operating point — while a traditional rack-average dashboard showed nothing wrong.

54 dayslead time · first signal → peak power
2.07×baseline power the node climbed to
4 / 847nodes flagged across the fleet
933 dayscontinuous telemetry analysed

The model walked Node 978 from STABLE → WATCH → DRIFTING → DEGRADING while its 16 rack-mates — sharing the same cooling, air, and workload pool — stayed flat. That divergence is the signal that separates a hardware fault from ordinary fleet-wide drift. Built on 698,461 daily power records.

GWDG · Göttingen, DE A100 GPU fleet · DCGM telemetry

GWDG — every NVIDIA A100 GPU failure with telemetry, caught early.

The same model, the same thresholds, pointed at a different cluster — GWDG's A100 GPU fleet — with no re-tuning. Scored forward-only against a year of operator-logged incidents as ground truth.

16 / 16telemetered GPU failures caught
4 daysmedian lead over the operator's report
15 / 16flagged before the human incident report
69incidents in the ground-truth catalog

Three forward-only layers — self-normalised behaviour, slow-drift slope, and DCGM hardware faults — score each node on telemetry alone. The detector ported straight from Marconi100 generalised to a new vendor and workload with zero domain tuning, confirming the approach isn't cluster-specific.

Bring it to your fleet

See it on your own fleet.

Send us a sample of your node telemetry and we'll build the same forensic case study on your hardware — or just tell us where node failures are costing you. Same model, same thresholds; no new sensors and no re-tuning between clusters.

Tell us about your cluster — the hardware, the workload, and the telemetry you already collect. We'll show you exactly where OrionLinks would surface node risk on your fleet.

Prefer to see it live? Book a walkthrough
Member of NVIDIA Inception Program member