CellType Launches BioSimBench: Frontier LLMs Get Only Part of Cell Responses Right, Gemini 4 Argon Leads and CellType 27B Beats Them on Every Track
CellType, a YC W26 startup, released BioSimBench 1.0 on November 4. It uses real lab experiments to test whether language models can predict which genes go up or down after a cell is hit by a drug, a gene knockdown or an immune signal. Among 12 frontier models, Gemini 4 Argon ranks first with an overall score of 0.184, ahead of Claude Sonnet 5.5 (0.160) and GPT-6 Astra (0.159), while two lab replicates agree at 0.462. A 27B model CellType trained in its BioSimGym RL environments beats the best frontier model on all four prediction tracks.
Key Takeaways
- BioSimBench 1.0 has five tracks: new drugs and cell contexts, gene knockdown, drug response, immune signals, and a tool-using research agent; test answers are kept private.
- The 12 frontier models score between 0.110 and 0.184 overall, with Gemini 4 Argon first and DeepSeek V4 Pro last; two lab replicates agree at 0.462, so the gap is still large.
- CellType 27B scores 0.366 on new drugs and cell contexts and 0.203 on drug response, about double the best frontier score of 0.103.
- Data processing uses NVIDIA RAPIDS-singlecell, at least 4x faster for the whole pipeline; models were trained with NeMo RL and BioSimGym follows NeMo RL environment interfaces.
Linked Models & Agents:
