![Grad-CAM autopsy at an amplitude of 4.0. (a) Q2_bright_hit: D1-3 S-shaped, detected. (b) Q2c_bright_mis: D1-1 extreme drift, missed. Columns: input, Grad-CAM map, and CAM difference (injected minus clean). Inputs are shown as the detector sees them: the three ON panels side by side, with frequency running vertically; white lines mark panel boundaries and the green line the injection’s starting channel. -- [astro-ph.IM]](https://i0.wp.com/astrobiology.com/wp-content/uploads/2026/10/Missed-Detections.png?fit=1200%2C679&ssl=1)
[astro-ph.IM] Radio technosignature searches rely on machine learning detectors to shrink the candidate pool before a human looks at it.
They are validated on held-out data from the training distribution, which excludes the cases where they are most likely to fail: a technosignature has unknown morphology, and terrestrial interference is unbounded in variety.
I trained a standard competition detector once, froze it, and probed it with 15,000 injections across ten morphologies and five amplitudes, placed into real Green Bank backgrounds.
Two directions were tested: genuine signals with morphologies other than a straight narrowband line, and interference resembling the training data. Recall did not follow the predicted ordering, and it fell as signals brightened, even for a needle of the kind the detector was trained on (from 0.590 to 0.090).
One interference morphology reached a false positive rate of 0.627, comparable to the detector’s own baseline recall of 0.5525 on genuine signals. Gradient-weighted Class Activation Mapping (Grad-CAM) showed attention along the injected path in one missed family by a pre-specified test, and visually in a second.
A pre-specified, time-averaged ON-OFF filter removed false alarms and genuine detections at the same rate. The detector reads only the ON panels.
Since the only evidence distinguishing a genuine candidate from terrestrial interference lies in the discarded OFF panels, it cannot tell if a signal originates from the target star; it can only tell if it resembles the training data.
This explains the false alarm directly; the missed signals reflect what the detector learned to recognise, which reading the OFF panels would not by itself change.
Omer Imran
Comments: 17 pages, 14 figures. Code and data: https://github.com/OmerImran/seti-two-directional-failure
Subjects: Instrumentation and Methods for Astrophysics (astro-ph.IM); Machine Learning (cs.LG); Popular Physics (physics.pop-ph)
Cite as: arXiv:2610.22050 [astro-ph.IM] (or arXiv:2610.22050v1 [astro-ph.IM] for this version)
https://doi.org/10.48550/arXiv.2610.22050
Focus to learn more
Submission history
From: Omer Imran
[v1] Thu, 15 Oct 2026 15:18:14 UTC (11,946 KB)
https://arxiv.org/abs/2610.22050
Astrobiology, SETI, Astronomy,
