Imaging & Spectroscopy

On The Application of Bayesian Leave-One-Out Cross-Validation to Exoplanet Atmospheric Analysis

By Keith Cowing

Press Release

astro-ph.EP

December 9, 2022

Filed under astro-ph.EP, astronomy, atmosphere, exoplanet, exoplanetary transmission spectra, extrasolar, imaging, Spectroscopy, telescope

On The Application of Bayesian Leave-One-Out Cross-Validation to Exoplanet Atmospheric Analysis — Comparison between the reference model (purple) and simpler models (green) with shaded regions showing their retrieved 2σ confidence intervals. The synthetic observations are color coded by their ∆elpd score following Equation 6, between the reference model and models without H2O (top left), Na (top right), K (bottom left), and inhomogeneous clouds and hazes (bottom right). Data points with larger positive ∆elpd scores (redder points) indicate that the reference model is better at explaining them. H2O absorption preferentially explains data points at ∼ 1.1-1.7µm, Na absorption at ∼ 0.6µm, and K absorption at ∼ 0.8µm, in agreement with expectations. The increase in the predictive performance of the model (the numerical scale of the color map) is largest due to H2O, followed by Na, and K; the inclusion of inhomogeneous clouds and hazes in the model does not improve the predictive performance of the model for this input cloud free atmosphere. — astro-ph.EP

Over the last decade, exoplanetary transmission spectra have yielded an unprecedented understanding about the physical and chemical nature of planets outside our solar system. Physical and chemical knowledge is mainly extracted via fitting competing models to spectroscopic data, based on some goodness-of-fit metric.

However, current employed metrics shed little light on how exactly a given model is failing at the individual data point level and where it could be improved. As the quality of our data and complexity of our models increases, there is an urgent need to better understand which observations are driving our model interpretations.

Here we present the application of Bayesian leave-one-out cross-validation to assess the performance of exoplanet atmospheric models and compute the expected log pointwise predictive density (elpdLOO). elpdLOO estimates the out-of-sample predictive accuracy of an atmospheric model at data point resolution providing interpretable model criticism. We introduce and demonstrate this method on synthetic HST transmission spectra of a hot Jupiter. We apply elpdLOO to interpret current observations of HAT-P-41b and assess the reliability of recent inferences of H− in its atmosphere.

We find that previous detections of H− are dependent solely on a single data point. This new metric for exoplanetary retrievals complements and expands our repertoire of tools to better understand the limits of our models and data. elpdLOO provides the means to interrogate models at the single data point level, a prerequisite for robustly interpreting the imminent wealth of spectroscopic information coming from JWST.

Luis Welbanks, Peter McGill, Michael Line, Nikku Madhusudhan

Comments: Accepted for publication in The Astronomical Journal
Subjects: Earth and Planetary Astrophysics (astro-ph.EP); Instrumentation and Methods for Astrophysics (astro-ph.IM)
Cite as: arXiv:2212.03872 [astro-ph.EP] (or arXiv:2212.03872v1 [astro-ph.EP] for this version)
Submission history
From: Luis Welbanks
[v1] Wed, 7 Dec 2022 19:00:00 UTC (2,737 KB)
https://arxiv.org/abs/2212.03872
Astrobiology,

Keith Cowing

Explorers Club Fellow, ex-NASA Space Station Payload manager/space biologist, Away Teams, Journalist, Lapsed climber, Synaesthete, Na’Vi-Jedi-Freman-Buddhist-mix, ASL, Devon Island and Everest Base Camp veteran, (he/him) 🖖🏻

Follow on Twitter