NASA Ames Machine Learning Seminar INDUS and INDUS-SDE: Scientific Language Models for Metadata Curation and Discovery Across NASA Science Divisions

JOIN THIS ML SEMINAR SERIES HERE

Time: October 27th, 12PM PDT/3PM EDT

Everyone is invited to the monthly machine learning virtual seminar hosted by the ML-Astrobiology Initiative at NASA Ames Research Center. Previously recorded seminars can be found on the ML-Astrobiology website.

The speaker for this event will be Dr. Kumar Ramasubramanian from the NASA-OSDI team and University of Alabama.

Title: INDUS and INDUS-SDE: Scientific Language Models for Metadata Curation and Discovery Across NASA Science Divisions

Abstract:

Scientific discovery in astrobiology depends on content drawn from planetary science, Earth science, biology, and astrophysics, each with its own terminology, and distributed across mission websites, data portals, technical reports, and publications with incomplete or inconsistent metadata.

General-purpose large language models do not resolve this fragmentation. Their answers are bounded by what their training data or retrieval layer happened to cover, and content that was never indexed or tagged remains invisible regardless of the interface used to query it. A query for all historical Raman spectroscopy data from Mars therefore returns a plausible subset with no guarantee of completeness.

We present INDUS, a family of compact encoder language models developed to address this problem at the level of the archive rather than the query. INDUS models are pretrained on NASA science text across five science divisions and are designed to curate heterogeneous scientific content at scale: assigning controlled-vocabulary keywords, classifying documents into topical portals, routing content to the appropriate science division, and producing embeddings for semantic retrieval. Because they are encoders, inference cost per document is low enough to process the full corpus continuously rather than sample from it.

The first generation of INDUS outperformed general-purpose and scientific baselines on NASA tasks and is deployed in NASA’s Science Discovery Engine. The second generation, INDUS-SDE, extends this work to the noisy, web-sourced content that constitutes most of the archive.

We introduce Weighted Dynamic Masking, a pretraining strategy that prioritizes domain-relevant terminology over structural boilerplate such as website headers, and an extended 1024-token context window suited to full metadata records. Evaluated on approximately 600,000 documents, INDUS-SDE matches domain-specific models trained on substantially more in-domain data, and its companion sentence transformer, INDUS-SDE-ST, supports semantic discovery over the curated corpus.

For astrobiology, this yields a shared representation space in which terminology from different disciplines aligns. In this talk, we discuss the utility of INDUS models for astrobiology, with examples of how they can be adapted to a new vocabulary and downstream task from a set of expert-labeled examples. We describe the deployed system, the requirements for such adaptation, and the role of community-contributed labeled data in extending INDUS to astrobiology-focused mission development.