A Blinded, Prospective Benchmark of In Silico Antibody Discovery Anchored to Experimental Affinity and Developability

M. Frank Erasmus¹, Daniel Bedinger², Elizabeth Hopkins³, Ginger Ferguson³, Justine Strickler³, Christilyn P. Graff⁴, Samantha R. Summers⁴, Stacy L. Capehart⁴, Joshua D. Slocum⁴, Crystal Richardson⁵, Sumit Kumar⁵, Zhifei Sun⁵, Yujie Shang⁵, Jixian Zhang⁶, Ming Gu⁶, Lixia Yi⁶, Alon Wellner⁶, Shuangjia Zheng⁶, Wei Lu⁶, Pietro Sormanni⁷, Matthew Greenig⁷, Haiping Zhang⁸, Brendan T. Mann⁹, Mahdi Baghbanzadeh⁹, Ali Rahnavard⁹, Gregory L. Moore¹⁰, Huaiyu Sun¹⁰, Ying Ding¹⁰, Alex Nisthal¹⁰, Jitendra Kanodia¹⁰, Matthew J. Bernett¹⁰, Aurélien Pélissier^11,12, Yanjun Shao¹¹, Maria Rodriguez Martinez¹¹, Karthik Ramesh¹³, Horacio Nastri¹³, Andreas Evers¹⁴, Anhar Abdelatif¹⁵, Andrew J. Bordner¹⁵, Mykola Bordyuh¹⁵, Lim Heo¹⁵, Brian A. Kidd¹⁵, H. Serhat Tetikol¹⁵, Shuai Wei¹⁵, Jung-Eun Shin¹⁶, Ryan Peckner¹⁶, Leigh Manley¹⁶, Ajitesh Lunge¹⁷, Yashas Devasurmutt¹⁷, Bora Guloglu¹⁸, Liviu Copoiu¹⁸, Miles McGibbon¹⁸, Monica L. Fernandez-Quintero¹⁹, Nitesh Mishra¹⁹, Sean M. Callaghan¹⁹, Olivia M. Swanson¹⁹, Daniel L. V. Bader¹⁹, James A. Ferguson¹⁹, Sai S. R. Raghavan¹⁹, Benjamin Nemoz¹⁹, Colleen A. Maillie¹⁹, Charles Bowman¹⁹, Bryan Briney¹⁹, Andrew B. Ward¹⁹, Paolo Marcatili²⁰, Rahmad Akbar²⁰, Bing He²¹, Fandi Wu²¹, Jianhua Yao²¹, Bih Hu²², Michal Kucer²², Kaetlyn Rose Gibson²², Rahul Somasundaram²², Li-Wei Hung²², Tomasz Kaszuba^23,24,25, Daved H. Fremont^{23,24,25,26,27}, Hyeongsun Jeong²⁸, Vinodh Babu Kurella²⁹, Shipra Malhotra²⁹, Satyendra Kumar²⁹, Yanyun Liu³⁰, Lingling Xu³⁰, Joshua Misa³⁰, Alexander Nicholas St. John³¹, Jeff Vogt³², Fátima A. Dávila-Hernández³², Da Xu³², Michael Chungyoung³², Zyaja D. Huggan³², Jeffrey J. Gray³², Jonathan Parkinson³³, Young Su Ko³³, Wei Wang³³, Franziska Geiger³⁴, Jonathon D. Ziegler³⁴, Nikhil Haas³⁵, Chance Challacombe³⁵, Ahmad Qamar³⁵, Akshita Singh³⁶, Yi-Ching Tang³⁶, Zhiqiang An³⁶, Xiaoqian Jiang³⁶, Yejin Kim³⁶, Xinyan Zhao³⁶, Erik Swanson³⁷, Jürgen Klattig³⁸, Karsten Winkler³⁸, Tschimegma Bataa³⁸, Volker Sandig³⁸, Lilian Denzler^12,39, Chunan Liu³⁹, Randall J. Brezski⁴⁰, Laura Spector¹, Katheryn Perea-Schmittle¹, Sara D’Angelo¹, Fortunato Ferrara¹ & Andrew R. M. Bradbury¹

Introduction

Experimentally validated prospective, blinded benchmarks are needed to separate durable advances from hype in computational antibody design. Here AIntibody, a challenge inspired by the Critical Assessment of Structure Prediction, tests 511 artificial intelligence (AI)-designed or predicted antibodies from 29 organizations on three tasks: in silico affinity maturation from phase 1 sequencing outputs, affinity ranking within heavy-chain complementarity-determining region 3 (HCDR3) clusters of a selection output and CDR design of proteins not included in a selection output. Validated with diverse experimental assays, several groups produced developable antibodies with affinities <100 pM. However, these successes were exceptions that did not transfer across tasks. Affinity-matured antibodies were modeled effectively. Except for one model, predicting high-affinity clones from clustered HCDR3 datasets was worse than random clone picking. Out-of-library design was highly variable for most method submissions, with many failing to outperform standard selections. The AIntibody challenge shows that AI can optimize antibodies in defined, biologically grounded regimes, in addition to highlighting critical gaps including affinity prediction and library-inspired antibody design and cross-task generalization.