[cs.LG] Foundation models trained on large unlabeled corpora develop emergent capabilities, tasks that go beyond the training objective.
We extend this paradigm to multimodal biology with Bonbon, a foundation model for molecular interactions trained on approximately 3.2 trillion tokens of paired protein-ligand sequences.
From a single representation space, Bonbon exhibits three zero-shot emergent capabilities at hierarchical levels of resolution: functional (mechanism of action), residue (binding site localization, including orthosteric/allosteric discrimination), and bond (covalent versus non-covalent engagement).
All three capabilities generalize to out-of-distribution complexes. Bonbon was trained on protein-ligand sequences alone, without any task-specific supervision and without structural coordinates.
We conducted controlled scaling experiments across six model scales from 42M to 1.6B parameters; every capability evaluated shows a sharp jump at the largest scale on continuous metrics, even as pretraining loss decreases smoothly. This demonstrates that loss-based scaling laws alone are insufficient to predict when zero-shot emergent capabilities become available.
The capabilities reported here emerge after training at a 1:2,000 parameter-to-token ratio to accommodate the sparsity of paired biological interaction data. The same frozen representations achieve state-of-the-art binding and affinity prediction at five orders of magnitude greater speed than co-folding methods.
In prospective wet-lab validation across nine disease targets spanning seven protein families, Bonbon achieves a 61.5% dose-response hit rate on de novo compounds, yielding structurally novel molecules not previously reported to inhibit these proteins, including for out-of-distribution targets.
Adi Mashiach
Comments: Accepted at NeurIPS 2026 (Main Track, Oral)
Subjects: Machine Learning (cs.LG); Biomolecules (q-bio.BM)
Cite as: arXiv:2611.14504 [cs.LG] (or arXiv:2611.14504v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2611.14504
Focus to learn more
Submission history
From: Adi Mashiach
[v1] Tue, 10 Nov 2026 18:53:01 UTC (9,592 KB)
https://arxiv.org/abs/2611.14504
Astrobiology,
